Mailing List Archive

> Note that it doesn't say "...followed by extra information... The
> extra information is sent as PATH_INFO, unless it happens to consist
> entirely of '/' characters, in which case we do some other thing with
> it".

I guess this is one level to far down, we need to see what the SSI
docs say about passing PATH_INFO from the current document (regular
html files don't have PATH_INFO), to the included cgi. I can't
reach the docs at the moment, but I'd be surprised if they explicitly
say that cgi includes assume the PATH_INFO of the parent document.

That's not to say they can't though.

> Is there a security hole here?

no. I shouldn't say that, I should say "I don't think so".

> If not, why is it "dangerous"? (I can
> see how it could be a little confusing, but only to people who brought
> it on themselves...).

That's the danger... It's going to confuse a hell of a lot of people,
because it's a half-baked system, with too many pitfalls. It only
makes sense to have the PATH_INFO, if the document has SSI, and the
SSI makes use of CGI - that narrows it down a bit.
Do we test for the other cases, and "404 Not Found" them ?

Nobody can be using this yet, 'cos NCSA 1.3/1.4 won't allow it. Is
it so desirable to have this, that we can live with the relative URL
problems that come with it ? I think not. 404 everything until HTTP/1.1
makes it fool proof.


robh

Date: Mon, 17 Apr 95 14:30 BST
From: drtr@ast.cam.ac.uk (David Robinson)

Rst wrote:
>...You'd need per-directory DefaultTypes (another potentially useful
>extension, though some care would be needed in implementation in the
>non-forking case)...

This is already available; it is a feature of NCSA httpd 1.3.

Hmmm... it's still there in 1.4, and still implemented as a simple
strcpy() into the default_type variable --- however, I can't see
anyplace where it saves the srm.conf DefaultType value before
overwriting it (and hence, it can't restore DefaultType to that value
before the next transaction).

(That's the subtlety I was alluding to... Rob H., if you've taken over
the non-forking stuff, I guess this is in your bailiwick).

> Re PATH_INFO; if /dir/file.ext is a regular (unix) file, then accessing
> /dir/file.ext/path_info will fail.
>
>Not currently --- the PATH_INFO is simply ignored in this case. I
>personally see no compelling reason to change this, although as we all
>will recall, Rob H. vehemently disagrees. However, I do think that
>PATH_INFO should clearly be allowed anywhere that a CGI script might
>get into the mix.

Urgle.
I think you did change this with B23.

Okay, I don't see any compelling reason to change it *back* ;-).

I based my comments on NCSA httpd 1.3, which has the following behaviour:
GET /index.html/a HTTP/1.0
returns 403 Forbidden
GET /index.html/ HTTP/1.0
returns index.html, but with Content-type: text/plain

Whereas for apache 0.5
GET /index.html/a HTTP/1.0
returns index.html
GET /index.html/ HTTP/1.0
returns index.html

The current apache behaviour is wrong.

Really? It is not at all obvious to me what the server should or
should not be doing in this case. In particular, the behavior you
propose below:

For /index.html/a it should give 404 Not Found.
For /index.html/ it should give 404 Not Found, or perhaps a redirect to
/index.html, as in general void path segments are not considered significant.

would *clearly* be improper with XBITHACK on --- if the XBIT is set on
/file.html/, then this is a reference to a server-side-includes file
with PATH_INFO, and the correct thing is very definitely to process
the file, and pass along the PATH_INFO to any scripts it happens to
invoke.

At any rate, this is such a minor issue that I can't see fussing with
it at all until after beta 1.

rst

> Nobody can be using this yet, 'cos NCSA 1.3/1.4 won't allow it. Is
> it so desirable to have this, that we can live with the relative URL
> problems that come with it ? I think not. 404 everything until HTTP/1.1
> makes it fool proof.

Incidentally, I dropped my proposal for a configurable setting to
"404 Not Found" directories that were missing a "/", after Roy
pointed out that HTTP/1.1 will offer a permanent solution.

I could live with the current Apache implementation (or whatever is
decided about how the variables are set), if it weren't documented.
Keep it quiet until 1.1 makes it bullet proof.

robh

>
> From: Rob Hartill <hartill@ooo.lanl.gov>
> Date: Mon, 17 Apr 95 10:03:53 MDT
>
> I guess this is one level to far down, we need to see what the SSI
> docs say about passing PATH_INFO from the current document (regular
> html files don't have PATH_INFO), to the included cgi. I can't
> reach the docs at the moment, but I'd be surprised if they explicitly
> say that cgi includes assume the PATH_INFO of the parent document.
>
> That's not to say they can't though.
>
> As I said, people are using it.

How can people be using it ?. 1.3 doesn't let you add path info
to say /index.html

I just tried it and got a "403 Forbidden" out of 1.3

Hmm, now that's a dumb response.

David wrote
> argumentative comments. Maybe it would do some good if I tried to write an
> actual specification.

That'd be useful to add to the Apache docs. If there's no alternative
out there, it might become the official spec - and you get to play
god :-)


robh

Date: Mon, 17 Apr 95 16:22 BST
From: drtr@ast.cam.ac.uk (David Robinson)

In fact, I don't understand why allowing PATH_INFO for server-side include
files is needed. I suspect it may be something to do with invoked CGI
script...

Precisely. There are a people with fairly elaborate setups which use
PATH_INFO to pass information around between scripts which are invoked
from different SSI pages.

Anyway, if it is, then the ssi documentation should emphasise the point
that you cannot use relative links without a BASE tag.

When we get around to documenting it ourselves, rather than borrowing
the NCSA docs, sure.

And earlier, Rst wrote:
>(Note that the server requires PATH_INFO of '/' when actually invoking
>httpd/unix-directory handlers --- sending out a redirect as usual
>otherwise --- so the handler can generate HTML which looks like:
> <a href="file_in_dir.foo">some file in this directory</a>
>without worry).

Actually, I think this is the wrong way of think about this case.
I would rather the DOCUMENT_URI ended in a '/', and the PATH_INFO was NULL.
A null PATH_INFO makes more sense.

Again, not to me. If you have a request for /script.cgi/foo/bar/zot,
then the PATH_INFO winds up as /foo/bar/zot. I am unable to see why
PATH_INFO of a single '/' should be treated any differently. In any
case, the current behavior is by far the simplest thing for anyone
actually *writing* an http/unix-directory handler (and yes, I've
written one to test the feature); I think that justifies it.

rst

Date: Mon, 17 Apr 95 16:30 BST
From: drtr@ast.cam.ac.uk (David Robinson)

Further thoughts, suppose I have a CGI script /cgi-bin/list and
an ssi /doc.shtml, what should happen in the event that I access
/doc.shtml/foo
which does
#exec cgi="/cgi-bin/list/bar"

should PATH_INFO for the CGI script be /foo or /bar ? Why would I ever want it
to be /foo ?

As explained earlier, you would want it to be /foo if your SSI
documents are using PATH_INFO to pass information around among
themselves (typically session-state information).

rst

From: Rob Hartill <hartill@ooo.lanl.gov>
Date: Mon, 17 Apr 95 9:10:30 MDT

> if the XBIT is set on
> /file.html/, then this is a reference to a server-side-includes file
> with PATH_INFO, and the correct thing is very definitely to process
^^^^^^^
> the file, and pass along the PATH_INFO to any scripts it happens to
> invoke.

I'll agree with that, if and only if you can point me to the CGI
documentation which defines this behaviour. If that isn't documented,
I'd agree that index.html/ and index.html/a should return a 404 Not
Found.

Here is what the CGI docs have to say about PATH_INFO:

<LI> <CODE>PATH_INFO</CODE> <P>
The extra path information, as given by the client. In other
words, scripts can be accessed by their virtual pathname, followed
by extra information at the end of this path. The extra
information is sent as PATH_INFO. This information should be
decoded by the server if it comes from a URL before it is passed
to the CGI script.<P>

Note that it doesn't say "...followed by extra information... The
extra information is sent as PATH_INFO, unless it happens to consist
entirely of '/' characters, in which case we do some other thing with
it".

Until we get HTTP/1.1, and the ability to add a BASE to the header, I
think it is too dangerous (w.r.t broken relative URLs) to service these
requests. If 1.3 behaves as David described, we won't be breaking
anything by 404'ing them.

Is there a security hole here? If not, why is it "dangerous"? (I can
see how it could be a little confusing, but only to people who brought
it on themselves...).

rst

Date: Mon, 17 Apr 95 16:39 BST
From: drtr@ast.cam.ac.uk (David Robinson)

I didn't mean in general, only for accessing a directory index.
I do not see that PATH_INFO would be relevent to an http/unix-directory
handler. But its not the value of PATH_INFO that I mind; rather that
if PATH_INFO has the '/', then DOCUMENT_URI does not, which is surely wrong.

*Surely* wrong? The semantics of httpd/unix-directory handlers aren't
even defined yet! If you have an argument to make as to why these
handlers should see a '/' on DOCUMENT_URI (as they presently don't),
then please make it, but ex cathedra statements like this aren't
terribly compelling.

I've got two arguments in favor of the status quo:

One is simple consistency with other handlers --- the implementation
would have to be special-cased to add the '/' for unix-directory
handlers, and I'd prefer to avoid that.

The other, which is somewhat more important, is that the major use for
DOCUMENT_URI in actually *writing* an httpd/unix-directory handler is
in generating headers --- and for headers you'd probably want to strip
the '/' off, if it were present, for the sake of neatness.

FYI, a simple httpd/unix-directory handler is included below (it's
actually a cut-down version of my test code); I'd prefer to keep the
minimal unix-directory handler at least this simple unless there's a
fairly compelling reason to do otherwise. (Yes, it does work, and the
relative URL's it generates are resolved correctly).

(If we were actually shipping $ENV{'DOCUMENT_URI'} values back to the
client in the http headers, I'd see more of an argument for setting
them "correctly", by your version of correctness --- we shouldn't be
referring clients to URLs which we will only redirect instantly. But
we aren't sending these values back to clients in the headers, so that
issue is moot. By the looks of things here, *very* moot).

rst




#! /usr/local/bin/perl

$dir_uri = $ENV{'DOCUMENT_URI'};
$dirname = $ENV{'DOCUMENT_TRANSLATED'};

opendir (dir, $dirname);
local (@files) = readdir(dir); closedir(dir);

print <<EOF;
Content-type: text/html

<Title>Index of $dir_uri</title>
<H1>Index of $dir_uri</h1>

<ul>
EOF

foreach $entry (@files)
{
print "<li> <a href=\"$entry\">$entry</a>\n";
}

print "</ul>\n";

From: Rob Hartill <hartill@ooo.lanl.gov>
Date: Mon, 17 Apr 95 10:03:53 MDT

I guess this is one level to far down, we need to see what the SSI
docs say about passing PATH_INFO from the current document (regular
html files don't have PATH_INFO), to the included cgi. I can't
reach the docs at the moment, but I'd be surprised if they explicitly
say that cgi includes assume the PATH_INFO of the parent document.

That's not to say they can't though.

As I said, people are using it.

> If not, why is it "dangerous"? (I can
> see how it could be a little confusing, but only to people who brought
> it on themselves...).

That's the danger... It's going to confuse a hell of a lot of people,
because it's a half-baked system, with too many pitfalls. It only
makes sense to have the PATH_INFO, if the document has SSI, and the
SSI makes use of CGI - that narrows it down a bit.
Do we test for the other cases, and "404 Not Found" them ?

Oh, what the heck. It turns out that there's an easy way to fix this
for the cases that people are worried about --- in send_file, add

if (path_args[0] != '\0') { ... die(...) }

*after* all the special-case checks for magic MIME types, but before
the fopen(). This wouldn't bother me too much --- what I'm really
worried about here is a B57-style "fix" for a very minor problem which
causes far more, and far nastier, problems than it solves due to
unforseen side effects, but that's not likely here.

rst

From: Rob Hartill <hartill@ooo.lanl.gov>
Date: Mon, 17 Apr 95 10:20:04 MDT

I could live with the current Apache implementation (or whatever is
decided about how the variables are set), if it weren't documented.
Keep it quiet until 1.1 makes it bullet proof.

Considering how shoddy we've been about documenting stuff which our
users might actually care about, I really don't think you have to
worry about anyone documenting *this* ;-).

rst

How can people be using it ?. 1.3 doesn't let you add path info
to say /index.html

Hmm... it *does* let you add PATH_INFO to *.shtml documents. It may
not allow you to add them to *.html (even with the x-bit set and XBITHACK),
but in that case, I think it probably should just for consistency.

rst

Date: Mon, 17 Apr 95 17:34 BST
From: drtr@ast.cam.ac.uk (David Robinson)
Precedence: bulk
Reply-To: new-httpd@hyperreal.com

I know...
My main problem is that I think the current ssi behaviour is completely
crap. The only spec is the source code itself, which is more broken than
a very broken thing, and upon which users probably depend. Hence my rather
argumentative comments. Maybe it would do some good if I tried to write an
actual specification.

OK, but given that users currently depend on the current behavior, I'm
not at all sure it's proper to change it incompatibly just because it
was never written down (even if you do think it's "completely crap").

Anyway, the only useful definitions of DOCUMENT_URI that I can think of are:
1. The path part of the URL for the requested resource.
2. The shortest URL formed by repeatedly removing path segments from the
path specified in 1 that would cause httpd to execute or parse the same
file (as for the full path).

It shouldn't be a surprise that I prefer 1.

Currently, for server-side includes, httpd implements some form of 2.
But I do think that for cgi scripts and handlers, knowning the actual URL
the user specified is important if you want to be able to return a document
with relative links. I would suggest that this is better to be completely
specified in the DOCUMENT_URI variable, than in the concatenation of
PATH_INFO and DOCUMENT_URI.

You can prefer that if you like --- but the current behavior parallels
the CGI case (users's full URL available only as the concatenation of
SCRIPT_NAME and PATH_INFO), and the CGI case really oughtn't change.

(Besides, in the handler context, which is what started this off, I
really think the current behavior is just more useful. A handler for
a discussion-form type system will probably want to do things like:

/some/forum/doc00001 --- the document

/some/forum/doc00001/followup_form --- a form which posts a
follow-up note

/some/forum/doc00001/post_followup --- URL which actually posts
the follow-up; this is why
POSTs to documents with
handlers work.

etc. The way DOCUMENT_URI is now defined for handlers, this works out
very conveniently for the author of the script --- the "command", if
any, is in PATH_INFO, and the relevant file is always named in
DOCUMENT_TRANSLATED, without any additional adornment that you'd have
to strip off. I've been thinking of actually implementing something
along these lines just so people could see how it fits together).

My original point was that, conceptually, I think about the URL for
a directory as having a trailing '/' as part of the resource name.
After all, I've been trained to think that way by the standard redirect
behaviour if the '/' is missing.

In fact, would you do a redirect for http://host/dir if it were handled
by a httpd/unix-directory handler? The CGI script could determine that the
user did not append a '/', and modify its relative links accordingly.

The current code does send these redirects even if a unix-directory
handler is defined --- that's why the relative URLs generated by the
sample handler I included in previous mail work correctly. (And yes,
that means that unix-directory handlers are *always* invoked with
PATH_INFO of '/'). I could be persuaded to change this, if I thought
it would actually make someone's job easier, but right now, it seems
to me that it goes more the other way...

rst

Rst wrote:
>...You'd need per-directory DefaultTypes (another potentially useful
>extension, though some care would be needed in implementation in the
>non-forking case)...

This is already available; it is a feature of NCSA httpd 1.3.

> Re PATH_INFO; if /dir/file.ext is a regular (unix) file, then accessing
> /dir/file.ext/path_info will fail.
>
>Not currently --- the PATH_INFO is simply ignored in this case. I
>personally see no compelling reason to change this, although as we all
>will recall, Rob H. vehemently disagrees. However, I do think that
>PATH_INFO should clearly be allowed anywhere that a CGI script might
>get into the mix.

Urgle.
I think you did change this with B23.

I based my comments on NCSA httpd 1.3, which has the following behaviour:
GET /index.html/a HTTP/1.0
returns 403 Forbidden
GET /index.html/ HTTP/1.0
returns index.html, but with Content-type: text/plain

Whereas for apache 0.5
GET /index.html/a HTTP/1.0
returns index.html
GET /index.html/ HTTP/1.0
returns index.html

The current apache behaviour is wrong.
For /index.html/a it should give 404 Not Found.
For /index.html/ it should give 404 Not Found, or perhaps a redirect to
/index.html, as in general void path segments are not considered significant.

David.

Rst wrote:
> /index.html, as in general void path segments are not considered significant.
>would *clearly* be improper with XBITHACK on --- if the XBIT is set on
>/file.html/, then this is a reference to a server-side-includes file
>with PATH_INFO, and the correct thing is very definitely to process
>the file, and pass along the PATH_INFO to any scripts it happens to
>invoke.

In fact, I don't understand why allowing PATH_INFO for server-side include
files is needed. I suspect it may be something to do with invoked CGI
script...
Anyway, if it is, then the ssi documentation should emphasise the point
that you cannot use relative links without a BASE tag.

And earlier, Rst wrote:
>(Note that the server requires PATH_INFO of '/' when actually invoking
>httpd/unix-directory handlers --- sending out a redirect as usual
>otherwise --- so the handler can generate HTML which looks like:
> <a href="file_in_dir.foo">some file in this directory</a>
>without worry).

Actually, I think this is the wrong way of think about this case.
I would rather the DOCUMENT_URI ended in a '/', and the PATH_INFO was NULL.
A null PATH_INFO makes more sense.

David.

RobH wrote:
>> if the XBIT is set on
>> /file.html/, then this is a reference to a server-side-includes file
>> with PATH_INFO, and the correct thing is very definitely to process
> ^^^^^^^
>> the file, and pass along the PATH_INFO to any scripts it happens to
>> invoke.
>
>I'll agree with that, if and only if you can point me to the CGI
>documentation which defines this behaviour. If that isn't documented,
>I'd agree that index.html/ and index.html/a should return a 404 Not
>Found.

Until we get HTTP/1.1, and the ability to add a BASE to the header, I
think it is too dangerous (w.r.t broken relative URLs) to service these
requests. If 1.3 behaves as David described, we won't be breaking
anything by 404'ing them.

rob

This probably sounds as though I didn't read what rst wrote, but anyway...
I wrote:
>Rst wrote:
>> /index.html, as in general void path segments are not considered significant.
>>would *clearly* be improper with XBITHACK on --- if the XBIT is set on
>>/file.html/, then this is a reference to a server-side-includes file
>>with PATH_INFO, and the correct thing is very definitely to process
>>the file, and pass along the PATH_INFO to any scripts it happens to
>>invoke.
>
>In fact, I don't understand why allowing PATH_INFO for server-side include
>files is needed. I suspect it may be something to do with invoked CGI
>script...

Further thoughts, suppose I have a CGI script /cgi-bin/list and
an ssi /doc.shtml, what should happen in the event that I access
/doc.shtml/foo
which does
#exec cgi="/cgi-bin/list/bar"

should PATH_INFO for the CGI script be /foo or /bar ? Why would I ever want it
to be /foo ?

David.

>Again, not to me. If you have a request for /script.cgi/foo/bar/zot,
>then the PATH_INFO winds up as /foo/bar/zot. I am unable to see why
>PATH_INFO of a single '/' should be treated any differently. In any
>case, the current behavior is by far the simplest thing for anyone
>actually *writing* an http/unix-directory handler (and yes, I've
>written one to test the feature); I think that justifies it.

I didn't mean in general, only for accessing a directory index.
I do not see that PATH_INFO would be relevent to an http/unix-directory
handler. But its not the value of PATH_INFO that I mind; rather that
if PATH_INFO has the '/', then DOCUMENT_URI does not, which is surely wrong.
(But maybe you don't have it working that way. Unfortunately, ssi files do.)

David.

>Precisely. There are a people with fairly elaborate setups which use
>PATH_INFO to pass information around between scripts which are invoked
>from different SSI pages.
But they shouldn't do! SSI should have a setenv for this purpose.
Of course, saying that doesn't do any good as we need to be compatible...

David.

Rst wrote:
>Here is what the CGI docs have to say about PATH_INFO:
>
><LI> <CODE>PATH_INFO</CODE> <P>
> The extra path information, as given by the client. In other
> words, scripts can be accessed by their virtual pathname, followed
> by extra information at the end of this path. The extra
> information is sent as PATH_INFO. This information should be
> decoded by the server if it comes from a URL before it is passed
> to the CGI script.<P>
>
I though RobH was asking about CGI scripts in the context of server-side
includes, which this does not mention.

>Note that it doesn't say "...followed by extra information... The
>extra information is sent as PATH_INFO, unless it happens to consist
>entirely of '/' characters, in which case we do some other thing with
>it".

And it doesn't say we couldn't....

In fact, treating null path segments specially would be quite legitimate.
httpd already imposes some semantics on the URL path, part of which
is sent to the CGI script as PATH_INFO. Consider: we parse '/../' in both the
document URI part and the PATH_INFO, removing it and the previous path
segment. Similarly, httpd ignores '//' in the doucment URI part, so maybe it
should also in the PATH_INFO ? 8-)

David.

>*Surely* wrong? The semantics of httpd/unix-directory handlers aren't
>even defined yet! If you have an argument to make as to why these
>handlers should see a '/' on DOCUMENT_URI (as they presently don't),
>then please make it, but ex cathedra statements like this aren't
>terribly compelling.

I know...
My main problem is that I think the current ssi behaviour is completely
crap. The only spec is the source code itself, which is more broken than
a very broken thing, and upon which users probably depend. Hence my rather
argumentative comments. Maybe it would do some good if I tried to write an
actual specification.

Anyway, what is DOCUMENT_URI anyway. The 'URI' for the document? To my mind,
that means the string passed in the request.

What does the SSI documentation say:
`DOCUMENT_URI: The virtual path to this document (such as /~robm/foo.shtml).'
* What is a virtual path? The documentation takes you to a description of the
path part of a URL.
* What does it mean _The_ virtual path to this document?
A document can have many URLs. Which one will httpd choose?
* What does it mean by 'this document'? Is the document not the resource that
the client receives? Or does it really mean 'this file'?

Anyway, the only useful definitions of DOCUMENT_URI that I can think of are:
1. The path part of the URL for the requested resource.
2. The shortest URL formed by repeatedly removing path segments from the
path specified in 1 that would cause httpd to execute or parse the same
file (as for the full path).

It shouldn't be a surprise that I prefer 1.

Currently, for server-side includes, httpd implements some form of 2.
But I do think that for cgi scripts and handlers, knowning the actual URL
the user specified is important if you want to be able to return a document
with relative links. I would suggest that this is better to be completely
specified in the DOCUMENT_URI variable, than in the concatenation of
PATH_INFO and DOCUMENT_URI.

My original point was that, conceptually, I think about the URL for
a directory as having a trailing '/' as part of the resource name.
After all, I've been trained to think that way by the standard redirect
behaviour if the '/' is missing.

In fact, would you do a redirect for http://host/dir if it were handled
by a httpd/unix-directory handler? The CGI script could determine that the
user did not append a '/', and modify its relative links accordingly.

David.

/*
* "Re: indexing suggestion (ATTN NCSA: possible 1.4 bug...)" by Brandon Long
* written Mon, 17 Apr 1995 10:29:23 -0500 (CDT)
*
* argh. You know, I'm beginning to understand why Netsite doesn't
* have local directory config files, just central ones that define
* everything. It makes it a lot easier.
*
*/

Actually, market demand made us add them to 1.1. So they're there,
improved, and with more sensible location methods than "stat the
whole damn tree". I think the real problem is that the httpd 1.3 code
makes so many assumptions about its own operation (no per-request
variable structure, glboals up the wazoo, die())

--Rob