Mailing List Archive

Dynamic DocumentRoots revisited
Okay. Thought I'd clarify some things from my post re Dynamic DocumentRoot.
Sorry, this is so long - but the subject seems to have generated a wave of
confusion out there. Although, ... this may not help...

I originally thought that I wanted to make server_rec's on the fly, but I
came to the conclusion that not only is that a pain in the rear-end but the
resources that are sucked up by additional server_rec's is definitely worse
that I would have figured it could be. I have everything I want covered for
my mass virtual hosting environment on Unix... all kinds of fun stuff for
the users, PHP, FrontPage Extensions for the Disciples of Bill, and various
& sundry services. The only thing left is this damned DOCUMENT_ROOT
environment variable which I'm finding that some common CGI's are relying on
- namely FrontPage Extension's counter program and some major
freeware/cheapware shopping cart programs that my users will want to use
until they realize they need something better.

Why do I need to set the DOCUMENT_ROOT? Because of the 65000 site this
service will run, probably 23000 of them will be FrontPage sites. That's
23000 *paying customers*. Everything else works for FrontPage - the
authoring, admin, etc. - everything except the counter which relies on the
env var DOCUMENT_ROOT. Naturally, I can rewrite the counter program - but
it points to me that other CGI's will have reliance on this particular
variable. I have seen it in the past five years with my own eyes. I'd like
to save my support group some phone calls.

No. Mod_rewrite and vhost_alias are not the magic answers to my prayers.
Both, unfortunately, are too simplistic (as blasphemous as that sounds) and
actually don't address the DOCUMENT_ROOT env var issue. I'm going to point
out some of these failings in hopes that Someone out there will see the
short sightedness of the vhost_alias module and put some additional efforts
into to true "mass virtual hosting":

mod_vhost_alias - okay great you can create a cookie cutter
structure with a few hundred site directories comfortably, a thousand
directories ... still usable but starting to get ugly .... two thousand
directories ... god, dont forget and do an ls ... but when I'm talking
65000, we're talking breakdown. So,... go to cookie cuttered subdirectory
structures... alpha? too many s's and w's, same problem as before ...
some type of other structure has to work ... yes, I've got it - it keeps
track of sites nicely - works with the auto-admin facilities (I'd tell you
all, but then I'd have to kill you...). So anyways, I have these structured
multi tiered directories that allow for international domains, domain
aliases, standard and nonstandard directories, that is also easy to
administer. I know how to tell apache where these are in the URI
Translation phase. No problems with the file translation phases and
everybody is happy with the exception of the dang document_root (which
mod_vhost_alias) doesn't address either.

mod_rewrite: Does not in anyway shape or form address the
document_root var issue. My URI translator is more effective anyways and
much more versitile ... and interacts nicely with my system that feeds me
the information I need to make the decisions ... without any need to reboot
the service.


Some people have said that I could introduce the document_root var at the
fixup stage. I haven't seen any modperl handlers that are native for this
process like the PerlTransHandler - although, I think I'm going blind and
it's probably there. I figure, that I'll have to inject the variable and do
a DECLINE so the other Fixup Handler processes can do what they need to do.
Has anybody played with this section?

As fellow unix geeks, you guys gotta help me out here. Otherwise, this damn
place is going to scrap the unix program and run everything on NT. I gotta
show them that NT doesn't stand a chance by blowing it completely away with
this platform. All I need is this one last piece!

---------------------------------------------------------------------
Best regards,

Karyn Ulriksen
Chief Systems Architect
PublicHost
22 Mauchly, Suite 200
Irvine, California 92618 USA
Phone: (949) 743-2000
email: kulriksen@publichost.com
URL: http://www.publichost.com
Re: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Some people have said that I could introduce the document_root var at the
> fixup stage. I haven't seen any modperl handlers that are native for this
> process like the PerlTransHandler - although, I think I'm going blind and
> it's probably there. I figure, that I'll have to inject the variable and do
> a DECLINE so the other Fixup Handler processes can do what they need to do.
> Has anybody played with this section?

I may be completely off base, because your explanation didn't tell me
everything I needed to know... but my whole application runs off a fixup
handler, so I know you can change things there and have stuff work:

$r->document_root($newroot);
$r->subprocess_env;
return OK;

--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Thanx for responding Matt.

Is the directive to identify this to Apache PerlFixupHandler or something?

-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 11:14 AM
To: Karyn Ulriksen; 'modperl@apache.org'
Subject: Re: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Some people have said that I could introduce the document_root var at the
> fixup stage. I haven't seen any modperl handlers that are native for this
> process like the PerlTransHandler - although, I think I'm going blind and
> it's probably there. I figure, that I'll have to inject the variable and
do
> a DECLINE so the other Fixup Handler processes can do what they need to
do.
> Has anybody played with this section?

I may be completely off base, because your explanation didn't tell me
everything I needed to know... but my whole application runs off a fixup
handler, so I know you can change things there and have stuff work:

$r->document_root($newroot);
$r->subprocess_env;
return OK;

--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
I could tell you how I know what to put in the document root var, but then I
would have to kill you....

You sound like you know something about unix so I'll answer your when with a
question. If you have 65000 directories in a subdirectory, wouldn't you
rather not? Scale, my man, Scale.

-Karyn

-----Original Message-----
From: Clifford Hammerschmidt [mailto:chammers@xaosnet.com]
Sent: Monday, February 28, 2000 11:16 AM
To: Karyn Ulriksen
Subject: Re: Dynamic DocumentRoots revisited


I may not grasp you problem fully but it really sounds like you need a C
handler that rewrites the ENV for you before mod_perl get's it chance to
take a whack at things. My question: how would it know what to put in the
DOCUMENT_ROOT? As long as you can map www.hostname.com to
/?/functionof(hostname)/htdocs/ or some such it should be doable.

>& sundry services. The only thing left is this damned DOCUMENT_ROOT
>environment variable which I'm finding that some common CGI's are relying
on
>- namely FrontPage Extension's counter program and some major
>freeware/cheapware shopping cart programs that my users will want to use
>until they realize they need something better.
RE: Dynamic DocumentRoots revisited [ In reply to ]
BTW, this is a prototype and C is obviously required for the long run, but I
want to understand this in Perl first... Poking at Apache internals is
somewhat new for me... -Karyn

-----Original Message-----
From: Karyn Ulriksen [mailto:kulriksen@publichost.com]
Sent: Monday, February 28, 2000 11:23 AM
To: 'Clifford Hammerschmidt'
Cc: 'modperl@apache.org'
Subject: RE: Dynamic DocumentRoots revisited


I could tell you how I know what to put in the document root var, but then I
would have to kill you....

You sound like you know something about unix so I'll answer your when with a
question. If you have 65000 directories in a subdirectory, wouldn't you
rather not? Scale, my man, Scale.

-Karyn

-----Original Message-----
From: Clifford Hammerschmidt [mailto:chammers@xaosnet.com]
Sent: Monday, February 28, 2000 11:16 AM
To: Karyn Ulriksen
Subject: Re: Dynamic DocumentRoots revisited


I may not grasp you problem fully but it really sounds like you need a C
handler that rewrites the ENV for you before mod_perl get's it chance to
take a whack at things. My question: how would it know what to put in the
DOCUMENT_ROOT? As long as you can map www.hostname.com to
/?/functionof(hostname)/htdocs/ or some such it should be doable.

>& sundry services. The only thing left is this damned DOCUMENT_ROOT
>environment variable which I'm finding that some common CGI's are relying
on
>- namely FrontPage Extension's counter program and some major
>freeware/cheapware shopping cart programs that my users will want to use
>until they realize they need something better.
RE: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Thanx for responding Matt.
>
> Is the directive to identify this to Apache PerlFixupHandler or something?

You got it. Actually my code below doesn't work, but this should:

$r->subprocess_env('DOCUMENT_ROOT', '/fred');
$r->subprocess_env;

I've been unable to test it as it doesn't seem to work with push_handlers,
but I'd be interested in you trying it.

> -----Original Message-----
> From: Matt Sergeant [mailto:matt@sergeant.org]
> Sent: Monday, February 28, 2000 11:14 AM
> To: Karyn Ulriksen; 'modperl@apache.org'
> Subject: Re: Dynamic DocumentRoots revisited
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > Some people have said that I could introduce the document_root var at the
> > fixup stage. I haven't seen any modperl handlers that are native for this
> > process like the PerlTransHandler - although, I think I'm going blind and
> > it's probably there. I figure, that I'll have to inject the variable and
> do
> > a DECLINE so the other Fixup Handler processes can do what they need to
> do.
> > Has anybody played with this section?
>
> I may be completely off base, because your explanation didn't tell me
> everything I needed to know... but my whole application runs off a fixup
> handler, so I know you can change things there and have stuff work:
>
> $r->document_root($newroot);
> $r->subprocess_env;
> return OK;
>
> --
> <Matt/>
>
> Details: FastNet Software Ltd - XML, Perl, Databases.
> Tagline: High Performance Web Solutions
> Web Sites: http://come.to/fastnet http://sergeant.org
> Available for Consultancy, Contracts and Training.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Thanx, I found PerlFixupHandler and will give it a whirl. I hope

$r->document_root($newroot);
$r->subprocess_env;

behaves better in the PerlFixupHandler than it does in the PerlTransHandler!
-Karyn

-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 11:14 AM
To: Karyn Ulriksen; 'modperl@apache.org'
Subject: Re: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Some people have said that I could introduce the document_root var at the
> fixup stage. I haven't seen any modperl handlers that are native for this
> process like the PerlTransHandler - although, I think I'm going blind and
> it's probably there. I figure, that I'll have to inject the variable and
do
> a DECLINE so the other Fixup Handler processes can do what they need to
do.
> Has anybody played with this section?

I may be completely off base, because your explanation didn't tell me
everything I needed to know... but my whole application runs off a fixup
handler, so I know you can change things there and have stuff work:

$r->document_root($newroot);
$r->subprocess_env;
return OK;

--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
Re: Dynamic DocumentRoots revisited [ In reply to ]
Hi again,

On Mon, 28 Feb 2000, Matt Sergeant wrote:

> $r->document_root($newroot);
> $r->subprocess_env;

Matt forgot to say, Eagle Book pp 454 - 457.

:)

73,
Ged.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Exactly! I'm not making ANY virtualhost entries in httpd.conf files. I
don't want to make 65000 VirtualHost entries that create server_recs that
suck up so many resources that there isn't much left over for serving. I am
already get the URI translation going where I want without any problems
through my own mechanism. I'm completely past the translation issue. Now,
I *simply* need to be able to set the document_root var for user CGI
processes. I'm playing with some stuff at the Fixup stage now. I'll let
you know if I get it going, so far it's just ignoring the variable when it's
set.

The setting of the DOCUMENT_ROOT var is *strictly* for CGI's only at this
point. Everything else is happy except the CGI's that use this var to make
decisions on file locations (which is very understandable).

-----Original Message-----
From: Clifford Hammerschmidt [mailto:chammers@xaosnet.com]
Sent: Monday, February 28, 2000 11:52 AM
To: Karyn Ulriksen
Subject: RE: Dynamic DocumentRoots revisited


Hmm...

Then you need an a pre-sorted structure to map from hostname -> /.../htdocs
...
If you can't map from SERVER_NAME or HTTP_HOST name to the htdocs path then
I have no idea.

A thought ... how are you getting all 65000 entries into the httpd.conf
file?

At 11:31 AM 2/28/2000 -0800, you wrote:
>mod_vhost_alias comes really close to this, but I would have to strictly
>limit my users ServerNames and ServerAlias to keep sanity - which would end
>up sending them packing to somewhere with less restrictions. No hampering
>is the rule of thumb at the ole web farm...
>
> -K
>
>-----Original Message-----
>From: Clifford Hammerschmidt [mailto:chammers@xaosnet.com]
>Sent: Monday, February 28, 2000 11:31 AM
>To: Karyn Ulriksen
>Subject: RE: Dynamic DocumentRoots revisited
>
>
>At 11:22 AM 2/28/2000 -0800, you wrote:
> >I could tell you how I know what to put in the document root var, but
then
>I
> >would have to kill you....
> >
> >You sound like you know something about unix so I'll answer your when
with
>a
> >question. If you have 65000 directories in a subdirectory, wouldn't you
> >rather not? Scale, my man, Scale.
>
>
>That's why it was functionof(hostname) and not hostname :) (eg,
>/com/x/a/o/s/n/e/t/htdocs ... ok, overkill... but you get the point.)
Re: Dynamic DocumentRoots revisited [ In reply to ]
On Feb 28, Karyn Ulriksen wrote:
> I could tell you how I know what to put in the document root var, but then I
> would have to kill you....

Still trying to figure out this open source thing, eh?

> You sound like you know something about unix so I'll answer your when with a
> question. If you have 65000 directories in a subdirectory, wouldn't you
> rather not? Scale, my man, Scale.

Then the real question is if you want to implement your hashing
at the application level, the operating system level, or somewhere
in between. (You could do it in the C library, even, and it would
be transparent to every application.)

The thread on the new-httpd mailing list contained a patch to
address that issue in mod_vhost_alias, and the discussion talked
about using something like MD5 or another hashing algorithm to
generate the tiers so you don't get the clustering you would see
with just generating them using substrings. From what Tony Finch
contributed to that thread, an approach like this is what Demon
Internet uses (and they were the ones who donated mod_vhost_alias
to Apache).

None of this is very complicated. 65000 users is peanuts. There's
people doing way more than that on Unix (www.homepage.com) and NT
(www.webjump.com). (Not to mention all the other free hosting
services out there -- who have other difficult problems you get to
avoid completely with paying customers.)

Jim
RE: Dynamic DocumentRoots revisited [ In reply to ]
Jim,

The hashing, getting the data I need, the translations .... These are all
there now. They have actually been there. I have the information to put
it. I have the RHS and LHS of the hash. I am just trying to plug it into
the environment. A simple $ENV{DOCUMENT_ROOT}="/some/dir/to/somewhere"
within the mod_perl context (so please don't leap all over this simple
syntax).

Maybe I should simply say:

Does anyone know where and how to set the DOCUMENT_ROOT variable so
that it passes into the CGI environment?



-----Original Message-----
From: Jim Winstead [mailto:jimw@trainedmonkey.com]
Sent: Monday, February 28, 2000 12:02 PM
To: Karyn Ulriksen
Cc: 'modperl@apache.org'
Subject: Re: Dynamic DocumentRoots revisited


On Feb 28, Karyn Ulriksen wrote:
> I could tell you how I know what to put in the document root var, but then
I
> would have to kill you....

Still trying to figure out this open source thing, eh?

> You sound like you know something about unix so I'll answer your when with
a
> question. If you have 65000 directories in a subdirectory, wouldn't you
> rather not? Scale, my man, Scale.

Then the real question is if you want to implement your hashing
at the application level, the operating system level, or somewhere
in between. (You could do it in the C library, even, and it would
be transparent to every application.)

The thread on the new-httpd mailing list contained a patch to
address that issue in mod_vhost_alias, and the discussion talked
about using something like MD5 or another hashing algorithm to
generate the tiers so you don't get the clustering you would see
with just generating them using substrings. From what Tony Finch
contributed to that thread, an approach like this is what Demon
Internet uses (and they were the ones who donated mod_vhost_alias
to Apache).

None of this is very complicated. 65000 users is peanuts. There's
people doing way more than that on Unix (www.homepage.com) and NT
(www.webjump.com). (Not to mention all the other free hosting
services out there -- who have other difficult problems you get to
avoid completely with paying customers.)

Jim
RE: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Jim,
>
> The hashing, getting the data I need, the translations .... These are all
> there now. They have actually been there. I have the information to put
> it. I have the RHS and LHS of the hash. I am just trying to plug it into
> the environment. A simple $ENV{DOCUMENT_ROOT}="/some/dir/to/somewhere"
> within the mod_perl context (so please don't leap all over this simple
> syntax).
>
> Maybe I should simply say:
>
> Does anyone know where and how to set the DOCUMENT_ROOT variable so
> that it passes into the CGI environment?

OK, I pass on how to get this working directly in mod_perl. I'm going to
suggest patching apache to do this...

So load up mod_cgi in your editor, and un-ifdef the bit in cgi_child() that
dumps the environment so you can see how the environment is structured. I
expect it's fairly simple. Then figure out a way to get your new
DOCUMENT_ROOT from r->notes which you'll fill with a PerlFixupHandler by
calling $r->notes('DOCUMENT_ROOT', '/new/root');

Good luck!

>
>
>
> -----Original Message-----
> From: Jim Winstead [mailto:jimw@trainedmonkey.com]
> Sent: Monday, February 28, 2000 12:02 PM
> To: Karyn Ulriksen
> Cc: 'modperl@apache.org'
> Subject: Re: Dynamic DocumentRoots revisited
>
>
> On Feb 28, Karyn Ulriksen wrote:
> > I could tell you how I know what to put in the document root var, but then
> I
> > would have to kill you....
>
> Still trying to figure out this open source thing, eh?
>
> > You sound like you know something about unix so I'll answer your when with
> a
> > question. If you have 65000 directories in a subdirectory, wouldn't you
> > rather not? Scale, my man, Scale.
>
> Then the real question is if you want to implement your hashing
> at the application level, the operating system level, or somewhere
> in between. (You could do it in the C library, even, and it would
> be transparent to every application.)
>
> The thread on the new-httpd mailing list contained a patch to
> address that issue in mod_vhost_alias, and the discussion talked
> about using something like MD5 or another hashing algorithm to
> generate the tiers so you don't get the clustering you would see
> with just generating them using substrings. From what Tony Finch
> contributed to that thread, an approach like this is what Demon
> Internet uses (and they were the ones who donated mod_vhost_alias
> to Apache).
>
> None of this is very complicated. 65000 users is peanuts. There's
> people doing way more than that on Unix (www.homepage.com) and NT
> (www.webjump.com). (Not to mention all the other free hosting
> services out there -- who have other difficult problems you get to
> avoid completely with paying customers.)
>
> Jim
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Hey Matt,

I tried a few different ways of setting this in the Fixup. Have you ever
actually tried to set this variable before? I can't believe I'm having this
much trouble with this one. I think I'll have to pull apart apache to see
if it's set read-only and why...

-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 11:26 AM
To: Karyn Ulriksen
Cc: 'modperl@apache.org'
Subject: RE: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Thanx for responding Matt.
>
> Is the directive to identify this to Apache PerlFixupHandler or something?

You got it. Actually my code below doesn't work, but this should:

$r->subprocess_env('DOCUMENT_ROOT', '/fred');
$r->subprocess_env;

I've been unable to test it as it doesn't seem to work with push_handlers,
but I'd be interested in you trying it.

> -----Original Message-----
> From: Matt Sergeant [mailto:matt@sergeant.org]
> Sent: Monday, February 28, 2000 11:14 AM
> To: Karyn Ulriksen; 'modperl@apache.org'
> Subject: Re: Dynamic DocumentRoots revisited
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > Some people have said that I could introduce the document_root var at
the
> > fixup stage. I haven't seen any modperl handlers that are native for
this
> > process like the PerlTransHandler - although, I think I'm going blind
and
> > it's probably there. I figure, that I'll have to inject the variable
and
> do
> > a DECLINE so the other Fixup Handler processes can do what they need to
> do.
> > Has anybody played with this section?
>
> I may be completely off base, because your explanation didn't tell me
> everything I needed to know... but my whole application runs off a fixup
> handler, so I know you can change things there and have stuff work:
>
> $r->document_root($newroot);
> $r->subprocess_env;
> return OK;
>
> --
> <Matt/>
>
> Details: FastNet Software Ltd - XML, Perl, Databases.
> Tagline: High Performance Web Solutions
> Web Sites: http://come.to/fastnet http://sergeant.org
> Available for Consultancy, Contracts and Training.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Hey Matt,
>
> I tried a few different ways of setting this in the Fixup. Have you ever
> actually tried to set this variable before? I can't believe I'm having this
> much trouble with this one. I think I'll have to pull apart apache to see
> if it's set read-only and why...

There's a set_document_root in http_core.c - I wonder if that's something
to do with it:

static const char *set_document_root(cmd_parms *cmd, void *dummy, char *arg)
{
void *sconf = cmd->server->module_config;
core_server_config *conf = ap_get_module_config(sconf, &core_module);

const char *err = ap_check_cmd_context(cmd,
NOT_IN_DIR_LOC_FILE|NOT_IN_LIMIT);
if (err != NULL) {
return err;
}

arg = ap_os_canonical_filename(cmd->pool, arg);
if (ap_configtestonly && ap_docrootcheck && !ap_is_directory(arg)) {
if (cmd->server->is_virtual) {
fprintf(stderr, "Warning: DocumentRoot [%s] does not exist\n",
arg);
}
else {
return "DocumentRoot must be a directory";
}
}

conf->ap_document_root = arg;
return NULL;
}


>
> -----Original Message-----
> From: Matt Sergeant [mailto:matt@sergeant.org]
> Sent: Monday, February 28, 2000 11:26 AM
> To: Karyn Ulriksen
> Cc: 'modperl@apache.org'
> Subject: RE: Dynamic DocumentRoots revisited
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > Thanx for responding Matt.
> >
> > Is the directive to identify this to Apache PerlFixupHandler or something?
>
> You got it. Actually my code below doesn't work, but this should:
>
> $r->subprocess_env('DOCUMENT_ROOT', '/fred');
> $r->subprocess_env;
>
> I've been unable to test it as it doesn't seem to work with push_handlers,
> but I'd be interested in you trying it.
>
> > -----Original Message-----
> > From: Matt Sergeant [mailto:matt@sergeant.org]
> > Sent: Monday, February 28, 2000 11:14 AM
> > To: Karyn Ulriksen; 'modperl@apache.org'
> > Subject: Re: Dynamic DocumentRoots revisited
> >
> >
> > On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > > Some people have said that I could introduce the document_root var at
> the
> > > fixup stage. I haven't seen any modperl handlers that are native for
> this
> > > process like the PerlTransHandler - although, I think I'm going blind
> and
> > > it's probably there. I figure, that I'll have to inject the variable
> and
> > do
> > > a DECLINE so the other Fixup Handler processes can do what they need to
> > do.
> > > Has anybody played with this section?
> >
> > I may be completely off base, because your explanation didn't tell me
> > everything I needed to know... but my whole application runs off a fixup
> > handler, so I know you can change things there and have stuff work:
> >
> > $r->document_root($newroot);
> > $r->subprocess_env;
> > return OK;
> >
> > --
> > <Matt/>
> >
> > Details: FastNet Software Ltd - XML, Perl, Databases.
> > Tagline: High Performance Web Solutions
> > Web Sites: http://come.to/fastnet http://sergeant.org
> > Available for Consultancy, Contracts and Training.
> --
> <Matt/>
>
> Details: FastNet Software Ltd - XML, Perl, Databases.
> Tagline: High Performance Web Solutions
> Web Sites: http://come.to/fastnet http://sergeant.org
> Available for Consultancy, Contracts and Training.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Matt,

You are truly a gentlemen and a scholar! Thanx for the no nonsense visit
on this issue. It's what I came here for and hope I can contribute. I'll
let you know how it works... -Karyn

-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 12:18 PM
To: Karyn Ulriksen; 'Jim Winstead'
Cc: 'modperl@apache.org'
Subject: RE: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Jim,
>
> The hashing, getting the data I need, the translations .... These are
all
> there now. They have actually been there. I have the information to put
> it. I have the RHS and LHS of the hash. I am just trying to plug it into
> the environment. A simple $ENV{DOCUMENT_ROOT}="/some/dir/to/somewhere"
> within the mod_perl context (so please don't leap all over this simple
> syntax).
>
> Maybe I should simply say:
>
> Does anyone know where and how to set the DOCUMENT_ROOT variable so
> that it passes into the CGI environment?

OK, I pass on how to get this working directly in mod_perl. I'm going to
suggest patching apache to do this...

So load up mod_cgi in your editor, and un-ifdef the bit in cgi_child() that
dumps the environment so you can see how the environment is structured. I
expect it's fairly simple. Then figure out a way to get your new
DOCUMENT_ROOT from r->notes which you'll fill with a PerlFixupHandler by
calling $r->notes('DOCUMENT_ROOT', '/new/root');

Good luck!

>
>
>
> -----Original Message-----
> From: Jim Winstead [mailto:jimw@trainedmonkey.com]
> Sent: Monday, February 28, 2000 12:02 PM
> To: Karyn Ulriksen
> Cc: 'modperl@apache.org'
> Subject: Re: Dynamic DocumentRoots revisited
>
>
> On Feb 28, Karyn Ulriksen wrote:
> > I could tell you how I know what to put in the document root var, but
then
> I
> > would have to kill you....
>
> Still trying to figure out this open source thing, eh?
>
> > You sound like you know something about unix so I'll answer your when
with
> a
> > question. If you have 65000 directories in a subdirectory, wouldn't you
> > rather not? Scale, my man, Scale.
>
> Then the real question is if you want to implement your hashing
> at the application level, the operating system level, or somewhere
> in between. (You could do it in the C library, even, and it would
> be transparent to every application.)
>
> The thread on the new-httpd mailing list contained a patch to
> address that issue in mod_vhost_alias, and the discussion talked
> about using something like MD5 or another hashing algorithm to
> generate the tiers so you don't get the clustering you would see
> with just generating them using substrings. From what Tony Finch
> contributed to that thread, an approach like this is what Demon
> Internet uses (and they were the ones who donated mod_vhost_alias
> to Apache).
>
> None of this is very complicated. 65000 users is peanuts. There's
> people doing way more than that on Unix (www.homepage.com) and NT
> (www.webjump.com). (Not to mention all the other free hosting
> services out there -- who have other difficult problems you get to
> avoid completely with paying customers.)
>
> Jim
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Well I think you are on to something because when I cracked open mod_cgi,
One of the first things after the standard heading is:

/*
* http_script: keeps all script-related ramblings together.
*
* Compliant to CGI/1.1 spec
*
* Adapted by rst from original NCSA code by Rob McCool
*
* Apache adds some new env vars; REDIRECT_URL and REDIRECT_QUERY_STRING for
* custom error responses, and DOCUMENT_ROOT because we found it useful.
* It also adds SERVER_ADMIN - useful for scripts to know who to mail when
* they fail.
*/

Since SERVER_ADMIN was the other variable I'm not able to set, I believe I
at least have a trail to follow now...

If I can swing this this, I can set a env var (ie "VIRTUAL_ROOT") in the
Translate phase, recall it in the fixup and apply it to DOCUMENT_ROOT saving
the NOTES section, since this other pool is already sitting there... make
sense?

As far as the function below, it's referring to the sconf which I believe
talks to the server_rec's, which I am trying to avoid (the little piglets
that they are). I wonder if this just is used by the server_rec pools?
I'll look around some, thanx.

-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 12:29 PM
To: Karyn Ulriksen
Cc: 'modperl@apache.org'
Subject: RE: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Hey Matt,
>
> I tried a few different ways of setting this in the Fixup. Have you
ever
> actually tried to set this variable before? I can't believe I'm having
this
> much trouble with this one. I think I'll have to pull apart apache to see
> if it's set read-only and why...

There's a set_document_root in http_core.c - I wonder if that's something
to do with it:

static const char *set_document_root(cmd_parms *cmd, void *dummy, char *arg)
{
void *sconf = cmd->server->module_config;
core_server_config *conf = ap_get_module_config(sconf, &core_module);

const char *err = ap_check_cmd_context(cmd,

NOT_IN_DIR_LOC_FILE|NOT_IN_LIMIT);
if (err != NULL) {
return err;
}

arg = ap_os_canonical_filename(cmd->pool, arg);
if (ap_configtestonly && ap_docrootcheck && !ap_is_directory(arg)) {
if (cmd->server->is_virtual) {
fprintf(stderr, "Warning: DocumentRoot [%s] does not exist\n",
arg);
}
else {
return "DocumentRoot must be a directory";
}
}

conf->ap_document_root = arg;
return NULL;
}


>
> -----Original Message-----
> From: Matt Sergeant [mailto:matt@sergeant.org]
> Sent: Monday, February 28, 2000 11:26 AM
> To: Karyn Ulriksen
> Cc: 'modperl@apache.org'
> Subject: RE: Dynamic DocumentRoots revisited
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > Thanx for responding Matt.
> >
> > Is the directive to identify this to Apache PerlFixupHandler or
something?
>
> You got it. Actually my code below doesn't work, but this should:
>
> $r->subprocess_env('DOCUMENT_ROOT', '/fred');
> $r->subprocess_env;
>
> I've been unable to test it as it doesn't seem to work with push_handlers,
> but I'd be interested in you trying it.
>
> > -----Original Message-----
> > From: Matt Sergeant [mailto:matt@sergeant.org]
> > Sent: Monday, February 28, 2000 11:14 AM
> > To: Karyn Ulriksen; 'modperl@apache.org'
> > Subject: Re: Dynamic DocumentRoots revisited
> >
> >
> > On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> > > Some people have said that I could introduce the document_root var at
> the
> > > fixup stage. I haven't seen any modperl handlers that are native for
> this
> > > process like the PerlTransHandler - although, I think I'm going blind
> and
> > > it's probably there. I figure, that I'll have to inject the variable
> and
> > do
> > > a DECLINE so the other Fixup Handler processes can do what they need
to
> > do.
> > > Has anybody played with this section?
> >
> > I may be completely off base, because your explanation didn't tell me
> > everything I needed to know... but my whole application runs off a fixup
> > handler, so I know you can change things there and have stuff work:
> >
> > $r->document_root($newroot);
> > $r->subprocess_env;
> > return OK;
> >
> > --
> > <Matt/>
> >
> > Details: FastNet Software Ltd - XML, Perl, Databases.
> > Tagline: High Performance Web Solutions
> > Web Sites: http://come.to/fastnet http://sergeant.org
> > Available for Consultancy, Contracts and Training.
> --
> <Matt/>
>
> Details: FastNet Software Ltd - XML, Perl, Databases.
> Tagline: High Performance Web Solutions
> Web Sites: http://come.to/fastnet http://sergeant.org
> Available for Consultancy, Contracts and Training.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:

> I tried a few different ways of setting this in the Fixup. Have you ever
> actually tried to set this variable before? I can't believe I'm having this
> much trouble with this one. I think I'll have to pull apart apache to see
> if it's set read-only and why...

I'm also curious why $r->document_root() is read-only.

Eagle book page 456 says "Apache uses the document root to translate the URI
into a physical pathname" but if you've rolled your own TransHandler, then
does it still matter? Certainly by the Fixup stage Apache shouldn't care
what DOCUMENT_ROOT is anymore, but (at least in my tests) it insists
on reverting to its set-in-stone value before the content generation phase.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Thank you. Thank you. Thank you. Thank you. Thank you.

I thought I was loosing my mind and everyone else was trying to push me over
the cliff.

Matt Sergeant has been pretty good about looking into this. I think we'll
all know how to handle this pretty soon. Optimistic? Yes!!

-Karyn

-----Original Message-----
From: Dan Rench [mailto:drench@i-works.com]
Sent: Monday, February 28, 2000 12:45 PM
To: modperl@apache.org
Subject: RE: Dynamic DocumentRoots revisited



On Mon, 28 Feb 2000, Karyn Ulriksen wrote:

> I tried a few different ways of setting this in the Fixup. Have you
ever
> actually tried to set this variable before? I can't believe I'm having
this
> much trouble with this one. I think I'll have to pull apart apache to see
> if it's set read-only and why...

I'm also curious why $r->document_root() is read-only.

Eagle book page 456 says "Apache uses the document root to translate the URI
into a physical pathname" but if you've rolled your own TransHandler, then
does it still matter? Certainly by the Fixup stage Apache shouldn't care
what DOCUMENT_ROOT is anymore, but (at least in my tests) it insists
on reverting to its set-in-stone value before the content generation phase.
RE: Dynamic DocumentRoots revisited [ In reply to ]
On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Thank you. Thank you. Thank you. Thank you. Thank you.
>
> I thought I was loosing my mind and everyone else was trying to push me over
> the cliff.
>
> Matt Sergeant has been pretty good about looking into this. I think we'll
> all know how to handle this pretty soon. Optimistic? Yes!!

Thought I'd do the work for you...

ap_add_cgi_vars(r);
ap_table_set(r->subprocess_env, "DOCUMENT_ROOT", ap_table_get(r->notes, "DOCUMENT_ROOT"));
env = ap_create_environment(r->pool, r->subprocess_env);

The middle line is the one I added - this is in mod_cgi.c, cgi_child().

>
> -Karyn
>
> -----Original Message-----
> From: Dan Rench [mailto:drench@i-works.com]
> Sent: Monday, February 28, 2000 12:45 PM
> To: modperl@apache.org
> Subject: RE: Dynamic DocumentRoots revisited
>
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
>
> > I tried a few different ways of setting this in the Fixup. Have you
> ever
> > actually tried to set this variable before? I can't believe I'm having
> this
> > much trouble with this one. I think I'll have to pull apart apache to see
> > if it's set read-only and why...
>
> I'm also curious why $r->document_root() is read-only.
>
> Eagle book page 456 says "Apache uses the document root to translate the URI
> into a physical pathname" but if you've rolled your own TransHandler, then
> does it still matter? Certainly by the Fixup stage Apache shouldn't care
> what DOCUMENT_ROOT is anymore, but (at least in my tests) it insists
> on reverting to its set-in-stone value before the content generation phase.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
RE: Dynamic DocumentRoots revisited [ In reply to ]
Matt Sergeant,

YOU ARE A GOD!!!!! It works! I bow to your greater glory. I humbly
lick your feet!!!

Did I mention, Thank you!? If you are ever in Southern California ...



-----Original Message-----
From: Matt Sergeant [mailto:matt@sergeant.org]
Sent: Monday, February 28, 2000 1:09 PM
To: Karyn Ulriksen; 'Dan Rench'
Cc: 'modperl@apache.org'
Subject: RE: Dynamic DocumentRoots revisited


On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
> Thank you. Thank you. Thank you. Thank you. Thank you.
>
> I thought I was loosing my mind and everyone else was trying to push me
over
> the cliff.
>
> Matt Sergeant has been pretty good about looking into this. I think we'll
> all know how to handle this pretty soon. Optimistic? Yes!!

Thought I'd do the work for you...

ap_add_cgi_vars(r);
ap_table_set(r->subprocess_env, "DOCUMENT_ROOT", ap_table_get(r->notes,
"DOCUMENT_ROOT"));
env = ap_create_environment(r->pool, r->subprocess_env);

The middle line is the one I added - this is in mod_cgi.c, cgi_child().

>
> -Karyn
>
> -----Original Message-----
> From: Dan Rench [mailto:drench@i-works.com]
> Sent: Monday, February 28, 2000 12:45 PM
> To: modperl@apache.org
> Subject: RE: Dynamic DocumentRoots revisited
>
>
>
> On Mon, 28 Feb 2000, Karyn Ulriksen wrote:
>
> > I tried a few different ways of setting this in the Fixup. Have you
> ever
> > actually tried to set this variable before? I can't believe I'm having
> this
> > much trouble with this one. I think I'll have to pull apart apache to
see
> > if it's set read-only and why...
>
> I'm also curious why $r->document_root() is read-only.
>
> Eagle book page 456 says "Apache uses the document root to translate the
URI
> into a physical pathname" but if you've rolled your own TransHandler, then
> does it still matter? Certainly by the Fixup stage Apache shouldn't care
> what DOCUMENT_ROOT is anymore, but (at least in my tests) it insists
> on reverting to its set-in-stone value before the content generation
phase.
--
<Matt/>

Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
Re: Dynamic DocumentRoots revisited [ In reply to ]
Karyn Ulriksen - Mon, Feb 28, 2000 at 10:54:29AM -0800

> mod_vhost_alias - okay great you can create a cookie cutter
> structure with a few hundred site directories comfortably, a thousand
> directories ... still usable but starting to get ugly .... two thousand
> directories ... god, dont forget and do an ls ... but when I'm talking
> 65000, we're talking breakdown. So,... go to cookie cuttered subdirectory
> structures... alpha? too many s's and w's, same problem as before ...
> some type of other structure has to work ... yes, I've got it - it keeps
> track of sites nicely - works with the auto-admin facilities (I'd
> tell you

The reiser file system a journaling filesystem for Linux does a nice
job with 10 of thousands of files in a directory. Not sure if it fits
your architecture but is simpler than a sub-directory hashing scheme.


--
Geoff Crawshaw -- gcrawshaw@timebills.com -- CTO
TimeBills.com -- Puts time on your side!
Track time, create invoices and manage expense reporting ... all on the Web.
Open your free account today at http://www.timebills.com
RE: Dynamic DocumentRoots revisited [ In reply to ]
Geoff,

Thanx for the input. Since the file systems will live on a toaster that
will be serving Unix and NT, I need to keep with the basics. But I'd still
be interested in hearing more on this journalizing file system. If you
could send me a link or some more info directly to me (since it's OT), I
would appreciate it. -Karyn

-----Original Message-----
From: Geoff Crawshaw [mailto:gcrawshaw@timebills.com]
Sent: Monday, February 28, 2000 4:29 PM
To: Karyn Ulriksen
Cc: 'modperl@apache.org'
Subject: Re: Dynamic DocumentRoots revisited


Karyn Ulriksen - Mon, Feb 28, 2000 at 10:54:29AM -0800

> mod_vhost_alias - okay great you can create a cookie cutter
> structure with a few hundred site directories comfortably, a thousand
> directories ... still usable but starting to get ugly .... two thousand
> directories ... god, dont forget and do an ls ... but when I'm talking
> 65000, we're talking breakdown. So,... go to cookie cuttered subdirectory
> structures... alpha? too many s's and w's, same problem as before ...
> some type of other structure has to work ... yes, I've got it - it keeps
> track of sites nicely - works with the auto-admin facilities (I'd
> tell you

The reiser file system a journaling filesystem for Linux does a nice
job with 10 of thousands of files in a directory. Not sure if it fits
your architecture but is simpler than a sub-directory hashing scheme.


--
Geoff Crawshaw -- gcrawshaw@timebills.com -- CTO
TimeBills.com -- Puts time on your side!
Track time, create invoices and manage expense reporting ... all on the Web.
Open your free account today at http://www.timebills.com