Mailing List Archive

ppp (was Re: Source Filters, the next generation)
>>>>> "Paul" == Paul Marquess <pmarquess@bfsec.bt.co.uk> writes:

>> > So... what do you think about the idea of having a ppp? I
>> *might* be > prepared to give this a go, *if* there is enough
>> interest. > Yes please. Certainly doing #define's, #if's and
>> #ifdef's should be straightforward. Doing macro substitutions with
>> parameters might be a little more tricky!

Paul> *If* I do it, it would only be #if #elsif #else #endif #define
Paul> and #undef. I wouldn't even attempt macros with parameters.

>> But why do it in perl? Why not find a public domain CPP and
>> include that? Much less effort and faster too.

I spent a few hours thinking about ppp, and it turns out things are
not really easy. Main problem: what should a "ppp" really do ?

1. If it is cpp-like, where should substitutions occur?

Say, I define
#define PI 3.141592653589793

Simple: $pi=PI;
Harder: $PI=PI;
Harder: print "The value of PI is $pi";

If you decided that, next is:

print <<EOT;
The value of PI is $PI
EOT

And then @constants=qw( E PI GAMMA );

I am not even decided what *I* presume should happen in these cases.

A PD cpp would have to be tuned up quite a bit, I fear.

--
Jost Krieger, Rechenzentrum der Ruhr-Universitaet Bochum
Jost.Krieger@rz.ruhr-uni-bochum.de
C=de;AD=d400;PD=ruhr-uni-bochum;OU=rz;OU=ruba;S=Krieger;G=Jost;
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
> From: Jost Krieger <x920031@hp138.rz.ruhr-uni-bochum.de>
>
> >>>>> "Paul" == Paul Marquess <pmarquess@bfsec.bt.co.uk> writes:
>
> >> > So... what do you think about the idea of having a ppp? I
> >> *might* be > prepared to give this a go, *if* there is enough
> >> interest. > Yes please. Certainly doing #define's, #if's and
> >> #ifdef's should be straightforward. Doing macro substitutions with
> >> parameters might be a little more tricky!
>
> Paul> *If* I do it, it would only be #if #elsif #else #endif #define
> Paul> and #undef. I wouldn't even attempt macros with parameters.
>
> >> But why do it in perl? Why not find a public domain CPP and
> >> include that? Much less effort and faster too.
>
> I spent a few hours thinking about ppp, and it turns out things are
> not really easy. Main problem: what should a "ppp" really do ?
>
Simple.

It should do *exactly* what CPP, and hence the perl -P flag, does
(or at least a strict subset).

I think a PD CPP is the best bet by far.

Tim.
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
> From: Nick.Ing-Simmons@tiuk.ti.com
>
> Yes a CPP would not know perl's syntax, and even if we did our own
> I can see 'ppp' vs perl being cause of FAQs for some while. I really think
> it would be easier to add 'const' to perl
>
It would be handy but wouldn't address #ifdefs or parameterised macros.

Tim.
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
>From: Jost Krieger <x920031@hp138.rz.ruhr-uni-bochum.de>
>>>>>> "Paul" == Paul Marquess <pmarquess@bfsec.bt.co.uk> writes:
> >> > So... what do you think about the idea of having a ppp? I
> >> *might* be > prepared to give this a go, *if* there is enough
> >> interest. > Yes please. Certainly doing #define's, #if's and
> >> #ifdef's should be straightforward. Doing macro substitutions with
> >> parameters might be a little more tricky!
> Paul> *If* I do it, it would only be #if #elsif #else #endif #define
> Paul> and #undef. I wouldn't even attempt macros with parameters.
> >> But why do it in perl? Why not find a public domain CPP and
> >> include that? Much less effort and faster too.
>I spent a few hours thinking about ppp, and it turns out things are
>not really easy. Main problem: what should a "ppp" really do ?

One real problem is that a ANSI-C conformant cpp is completely unusable
on perl code. The ANSI-C standard insists that a fully-comformant cpp
tokenize the source using into a C style token stream and certain perl
syntax completely confuse that. For Example, C comments vs perl comments,
the use of ilegal-outside-of-C-comments characters in perl ($, @, #, `),
old version main'foo names with unbalanced ', perl allows real newlines
embedded in strings C doesn't, etc. Much as it would be nice to be able
to use what ever local cpp you have as ppp it just doesn't work. As another
example the -P switch has never been really usable with the various DOS versions
of perl because both the Borland and Microsoft C ccp are ANSI-conformant and
blow sky high on most perl code.

--
Mark Biggar
mab@wdl.loral.com
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
> From: Paul Marquess <pmarquess@bfsec.bt.co.uk>
>
> From: Tim Bunce <Tim.Bunce@ig.co.uk>
> >
> > > From: Jost Krieger <x920031@hp138.rz.ruhr-uni-bochum.de>
> > >
> > > >>>>> "Paul" == Paul Marquess <pmarquess@bfsec.bt.co.uk> writes:
> > >
> > > >> > So... what do you think about the idea of having a ppp? I
> > > >> *might* be > prepared to give this a go, *if* there is enough
> > > >> interest. > Yes please. Certainly doing #define's, #if's and
> > > >> #ifdef's should be straightforward. Doing macro substitutions with
> > > >> parameters might be a little more tricky!
> > >
> > > Paul> *If* I do it, it would only be #if #elsif #else #endif #define
> > > Paul> and #undef. I wouldn't even attempt macros with parameters.
> > >
> > > >> But why do it in perl? Why not find a public domain CPP and
> > > >> include that? Much less effort and faster too.
> > >
> > > I spent a few hours thinking about ppp, and it turns out things are
> > > not really easy. Main problem: what should a "ppp" really do ?
> > >
> > Simple.
> >
> > It should do *exactly* what CPP, and hence the perl -P flag, does
> > (or at least a strict subset).
>
> As Perl obviously isn't C, should ppp mirror cpp as closely as possible?
>
> *If* ppp was written from scratch we could start with a clean slate and
> define a pre-processor which was appropriate for Perl.
>
There are two separate requirements being mixed up here:

1) Allow perl -P to work on all platforms.
2) Provide a preprocessor that understands perl syntax better.

1) is the real requirement and relates to the defined meaning of the -P
flag and to the portability of existing code written to use -P.

2) is very much in the 'might be nice to have' category.

Calling (1) 'ppp' is probably not helping.

> > I think a PD CPP is the best bet by far.
>
> As long as a sub-process filter is used it would be quite easy to get
> a PD CPP to work.
>
Sounds good.

> Paul
>
Tim.
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
On Thu, 24 Aug 1995, Tim Bunce wrote:

> There are two separate requirements being mixed up here:
>
> 1) Allow perl -P to work on all platforms.
> 2) Provide a preprocessor that understands perl syntax better.
>
> 1) is the real requirement and relates to the defined meaning of the -P
> flag and to the portability of existing code written to use -P.
>
> 2) is very much in the 'might be nice to have' category.

Well said. And further, I personally don't view 1) as a very high
priority. It's very hard to change "many platforms" to "almost all
platforms", and I'm not at all convinced it's worth the effort.

Andy Dougherty doughera@lafcol.lafayette.edu
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
From: Tim Bunce <Tim.Bunce@ig.co.uk>
>
>
> > From: Jost Krieger <x920031@hp138.rz.ruhr-uni-bochum.de>
> >
> > >>>>> "Paul" == Paul Marquess <pmarquess@bfsec.bt.co.uk> writes:
> >
> > >> > So... what do you think about the idea of having a ppp? I
> > >> *might* be > prepared to give this a go, *if* there is enough
> > >> interest. > Yes please. Certainly doing #define's, #if's and
> > >> #ifdef's should be straightforward. Doing macro substitutions with
> > >> parameters might be a little more tricky!
> >
> > Paul> *If* I do it, it would only be #if #elsif #else #endif #define
> > Paul> and #undef. I wouldn't even attempt macros with parameters.
> >
> > >> But why do it in perl? Why not find a public domain CPP and
> > >> include that? Much less effort and faster too.
> >
> > I spent a few hours thinking about ppp, and it turns out things are
> > not really easy. Main problem: what should a "ppp" really do ?
> >
> Simple.
>
> It should do *exactly* what CPP, and hence the perl -P flag, does
> (or at least a strict subset).

As Perl obviously isn't C, should ppp mirror cpp as closely as
possible?

*If* ppp was written from scratch we could start with a clean slate and
define a pre-processor which was appropriate for Perl.

> I think a PD CPP is the best bet by far.

As long as a sub-process filter is used it would be quite easy to get
a PD CPP to work.

Paul
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
In <9508241652.aa09814@post.demon.co.uk>
On Thu, 24 Aug 1995 16:55:07 +0100
Tim Bunce <Tim.Bunce@ig.co.uk> writes:
>> From: Nick.Ing-Simmons@tiuk.ti.com
>>
>> Yes a CPP would not know perl's syntax, and even if we did our own
>> I can see 'ppp' vs perl being cause of FAQs for some while. I really think
>> it would be easier to add 'const' to perl
>>
>It would be handy but wouldn't address #ifdefs or parameterised macros.

True, but as #ifdef is just a way of avoiding compiling something,
it could be considered as akin to :

BEGIN
{
if (defined $var)
{

}
}

Or perhaps even more like way perl ignores embedded pod stuff:

So how about

=if defined $var
=else
=endif

And do we really need parameterized macros?
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
> From: Andy Dougherty <doughera@lafcol.lafayette.edu>
>
> On Thu, 24 Aug 1995, Tim Bunce wrote:
>
> > There are two separate requirements being mixed up here:
> >
> > 1) Allow perl -P to work on all platforms.
> > 2) Provide a preprocessor that understands perl syntax better.
> >
> > 1) is the real requirement and relates to the defined meaning of the -P
> > flag and to the portability of existing code written to use -P.
> >
> > 2) is very much in the 'might be nice to have' category.
>
> Well said. And further, I personally don't view 1) as a very high
> priority. It's very hard to change "many platforms" to "almost all
> platforms", and I'm not at all convinced it's worth the effort.
>
Actually I agree with you. I was/am concerned that the requirement was
drifting without a clear focus.

Perhaps we should discuss the fate of the perl -P flag.

Should it be deprecated?

Should it invoke a new perl preprocessor?

Is it actually needed anymore since anyone writing a perl source file
needing a preprocessor could put 'use ppp;' into the code at the time.

Maybe getting -P to work should require a build option in the next
release so that we get an accurate idea of how many people use it.

> Andy Dougherty doughera@lafcol.lafayette.edu
>
Tim.
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
>>>>> "Tim" == Tim Bunce <Tim.Bunce@ig.co.uk> writes:
In article <9508241912.aa12027@post.demon.co.uk> Tim Bunce <Tim.Bunce@ig.co.uk> writes:
Tim> There are two separate requirements being mixed up here:

Tim> 1) Allow perl -P to work on all platforms. 2) Provide a
Tim> preprocessor that understands perl syntax better.

Tim> 1) is the real requirement and relates to the defined meaning of
Tim> the -P flag and to the portability of existing code written to
Tim> use -P.

Tim> 2) is very much in the 'might be nice to have' category.

Tim> Calling (1) 'ppp' is probably not helping.

The distinction is very clear, but I don't believe in the "real requirement".

Let's have a look: Who is using perl -P ? Speak up, please.

Especially with respect to the ANSI/non-ANSI problem,
perl -P seems not to be too well-defined, so the real solution might
be to "depreciate" it.

2) stays in the 'nice to have' category, but the only important aspect remaining
seem to be the "nonchangeable" variables, as neither #ifdef nor parameterized
macros are really helpful for Perl 5.

Jost

--
Jost Krieger, Rechenzentrum der Ruhr-Universitaet Bochum
Jost.Krieger@rz.ruhr-uni-bochum.de
C=de;AD=d400;PD=ruhr-uni-bochum;OU=rz;OU=ruba;S=Krieger;G=Jost;
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
What do we want from ppp?

1. (Parameterized) macros
2. Constants

Anything else?

For constants, a "const" keyword is probably easiest. I don't think
that it would take a great deal of work, given perl 5's current
handling of SV's (in fact, I don't know if it would take ANY work
other than adding the grammar, and setting the readonly flag).

Macros are another story. It would be comparatively easy to make
macros work very much like subroutines that take parameter lists:

macro foo ($a, $b, $c) {
return $a<=>$b || $c;
}

$result = foo($v1, $v2, $default);

In this example, the call to foo would be expanded into something
like:

{
local *a = \$v1; local *b = \$v2; local *c = \$default;
$result = $a<=>$b || $c;
}

Note that macros get all parameters passed-by-reference, so modifying
parameters is doable.

I think you can see how this transformation takes place. The hard part
is expressions and such. eg:

$result = foo($v1+7, $v2, <STDIN>);

This CAN be solved if we agree on two things:

1. Macros supply a scalar context to their parameter lists.
2. Macro calls MUST occur after perl sees the definition.

In this case, the example above simply allocates a static scoped
variable for the expressions that it cannot turn into an SV at
compile-time. eg:

{
my $a = $v1+7;
local *b = \$v2;
my $c = <STDIN>;
$result = $a<=>$b||$c;
}

If both of these features (a const keyword and a macro keyword) were
implimented, I don't think that there would be a need for a ppp.

Side question: Why does "my *a = \$b" fail? What is so special about
globbing that it can't be done statically.

-AJS
--
--- Aaron Sherman / "B4 f w+ c kv s+(--)v r p" ---
Phone: (617)321-5100 "I do not speak for THEM."
Email: ajs@ajs.com WWW: http://ajs.com/~ajs/
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
> From: Aaron Sherman <ajs@vorlon.ajs.com>
>
> What do we want from ppp?
>
> 1. (Parameterized) macros
> 2. Constants
>
> Anything else?
>
I hear the creaking sound of a can of worms being opened.

> Note that macros get all parameters passed-by-reference, so modifying
> parameters is doable.
>
Please, lets just stick to textual substition with a slightly improved
(more perl aware) tokeniser.

> -AJS

Tim.
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
>From: Aaron Sherman <ajs@vorlon.ajs.com>
>What do we want from ppp?
> 1. (Parameterized) macros
> 2. Constants
>Anything else?
>For constants, a "const" keyword is probably easiest. I don't think
>that it would take a great deal of work, given perl 5's current
>handling of SV's (in fact, I don't know if it would take ANY work
>other than adding the grammar, and setting the readonly flag).

Agreed.

>Macros are another story. It would be comparatively easy to make
>macros work very much like subroutines that take parameter lists:
>
> macro foo ($a, $b, $c) {
> return $a<=>$b || $c;
> }
>
> $result = foo($v1, $v2, $default);
>In this example, the call to foo would be expanded into something
>like:
>
> {
> local *a = \$v1; local *b = \$v2; local *c = \$default;
> $result = $a<=>$b || $c;
> }
> [rest deleted]

Note that we already can emulate lisp style macros. In lisp a macro is
just a compile time function whose result is inserted back into the parsing
token stream. A combination of BEGIN {}, a function call and eval() is all
that is needed to generate macroized subs. and if we had a version of eval()
with the one time only semantics of eval {}, it would be even simpler. I think
that what I'm trying to say is that perl already contains all the necessary
stuff to do lisp style macros, all we need is a little syntatic sugar.

--
Mark Biggar
mab@wdl.loral.com
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
>>>>> "Tim" == Tim Bunce <Tim.Bunce@ig.co.uk> writes:

Tim> Please, lets just stick to textual substition with a slightly
Tim> improved (more perl aware) tokeniser.

Fine. I have to admit that I'm too busy working on CORBA to do
anything with this, but I would at least consider a more LISP-like
model if I were you.

-AJS

--
--- Aaron Sherman / "B4 f w+ c kv s+(--)v r p" ---
Phone: (617)321-5100 "I do not speak for THEM."
Email: ajs@ajs.com WWW: http://ajs.com/~ajs/
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
Tim Bunce wrote :
|| > From: Aaron Sherman <ajs@vorlon.ajs.com>
|| >
|| > What do we want from ppp?
|| >
|| > 1. (Parameterized) macros
|| > 2. Constants
|| >
|| > Anything else?
|| >
|| I hear the creaking sound of a can of worms being opened.
||
|| > Note that macros get all parameters passed-by-reference, so modifying
|| > parameters is doable.
|| >
|| Please, lets just stick to textual substition with a slightly improved
|| (more perl aware) tokeniser.

I'd even avoid perl-aware as an issue. There are too many
different special cases within perl to even crack that can
(myriad quoting methods, formats, other filters) and it is not
always clear just which parts you would want to treat specially
in a language-dependent way within the filter anyhow - you might
want to filter the .pod text, for example.

If anyone is going to bother with a perl pre processor (which I
don't feel any significant need for - I've never used the -P
option, for example), I'd suggest the following attributes:
- line-oriented
- conditional inclusion/exclusion of lines
- textual substitution
- easy to specify whether substitutions applied for a
line/block of lines
Substitutions with arguments, and file inclusions are not
especially necessary (although file inclusion with "use",
"require", or "do" do not automatically pass through the pre
processor, and a separate filtering pass loses the current
textual context).

Something like "vc" could be about right. (I've only used vc
slightly, and was quite surprised when I originally found it.
It is quite wide-spread - it was part of the V7 SCCS suite -
from before the split between BSD and SYS III. My surprise was
that it was so wide-spread and useful, yet I hadn't noticed it
before. On the other hand, I still haven't used it all that
much.)

(Since much of the pre-processing can just be done within perl
itself, I am reminded of the PL/I preprocessor - the entire
language PL/I is available *at compile time* to compute the
program to be compiled. You can do things like initialize an
array with a complicated expression once at compile time, and
have just the value go into the loadable image. Yet another
reason for that masters project for a perl compiler.)

--
Maybe we can fix reality one of these days. | John Macdonald
<- Larry Wall Carl Dichter -> | jmm@Elegant.COM
I'd be happy if we could just index it. |
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
From: jmm@elegant.com (John Macdonald)
>
> Tim Bunce wrote :
> || > From: Aaron Sherman <ajs@vorlon.ajs.com>
> || >
> || > What do we want from ppp?
> || >
> || > 1. (Parameterized) macros
> || > 2. Constants
> || >
> || > Anything else?
> || >
> || I hear the creaking sound of a can of worms being opened.
> ||
> || > Note that macros get all parameters passed-by-reference, so modifying
> || > parameters is doable.
> || >
> || Please, lets just stick to textual substition with a slightly improved
> || (more perl aware) tokeniser.
>
> I'd even avoid perl-aware as an issue. There are too many
> different special cases within perl to even crack that can
> (myriad quoting methods, formats, other filters) and it is not
> always clear just which parts you would want to treat specially
> in a language-dependent way within the filter anyhow - you might
> want to filter the .pod text, for example.

I agree. To make ppp properly perl-aware you are really going to need
to parse perl.

> If anyone is going to bother with a perl pre processor (which I
> don't feel any significant need for - I've never used the -P
> option, for example), I'd suggest the following attributes:
> - line-oriented
> - conditional inclusion/exclusion of lines
> - textual substitution
> - easy to specify whether substitutions applied for a
> line/block of lines
> Substitutions with arguments, and file inclusions are not
> especially necessary (although file inclusion with "use",
> "require", or "do" do not automatically pass through the pre
> processor, and a separate filtering pass loses the current
> textual context).
>
> Something like "vc" could be about right. (I've only used vc
> slightly, and was quite surprised when I originally found it.
> It is quite wide-spread - it was part of the V7 SCCS suite -
> from before the split between BSD and SYS III. My surprise was
> that it was so wide-spread and useful, yet I hadn't noticed it
> before. On the other hand, I still haven't used it all that
> much.)
>
> (Since much of the pre-processing can just be done within perl
> itself, I am reminded of the PL/I preprocessor - the entire
> language PL/I is available *at compile time* to compute the
> program to be compiled. You can do things like initialize an
> array with a complicated expression once at compile time, and
> have just the value go into the loadable image. Yet another
> reason for that masters project for a perl compiler.)

I must admit that I've gone completely off the boil with the whole
issue of ppp. If anyone feels inclined to take it on, go right ahead.

Paul
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
Why does the PPP have to be separate from the interpreter? That was my
point in refering to the way that LISP does things.

--
--- Aaron Sherman / "B4 f w+ c kv s+(--)v r p" ---
Phone: (617)321-5100 "I do not speak for THEM."
Email: ajs@ajs.com WWW: http://ajs.com/~ajs/
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
: 2) stays in the 'nice to have' category, but the only important aspect remaining
: seem to be the "nonchangeable" variables, as neither #ifdef nor parameterized
: macros are really helpful for Perl 5.

Well, it'd be nice to be able to construct code at compile time using
functions that evaluate immediately. Presumably the output could be
slid back into the lexer as a subparse, the same way that double-quoted
strings and such are handled.

But yes, it's just a nice-to-have, unless you're trying to reinvent Scheme.

Larry
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
On Fri, 1 Sep 1995, Larry Wall wrote:

> : 2) stays in the 'nice to have' category, but the only important aspect remaining
> : seem to be the "nonchangeable" variables, as neither #ifdef nor parameterized
> : macros are really helpful for Perl 5.
>
> Well, it'd be nice to be able to construct code at compile time using
> functions that evaluate immediately. Presumably the output could be
> slid back into the lexer as a subparse, the same way that double-quoted
> strings and such are handled.
>
> But yes, it's just a nice-to-have, unless you're trying to reinvent Scheme.

Well, on that note...

How about a MEANWHILE? Something that would execute before code execution
but after BEGIN, and if the block returns anything it would get fed back
to the lexer.

> Larry

--
Kenneth Albanowski (kjahds@kjahds.com, CIS: 70705,126)
Re: ppp (was Re: Source Filters, the next generation) [ In reply to ]
Aaron Sherman wrote :
|| Subject: Re: ppp (was Re: Source Filters, the next generation)
||
|| Why does the PPP have to be separate from the interpreter? That was my
|| point in refering to the way that LISP does things.

It does not have to be. Source filters can be either embedded
code (either C or Perl) or an external program. The existing
use of the C pre-processor necessarily uses an external program,
but a alternative (not a total replacement) Perl pre-processor
would likely be embedded into the interpretor - either when it
was built or as a dynamically loadable module.

--
Maybe we can fix reality one of these days. | John Macdonald
<- Larry Wall Carl Dichter -> | jmm@Elegant.COM
I'd be happy if we could just index it. |