Mailing List Archive

On Mon, 6 Nov 1995, Tom Christiansen wrote:

> [. I notice only one chat2 bug, NETaa14400. I propose that this email
> be taken as a bug report entitled "upgrade chat2 to use Comm". ]

I have no objection, but I'd appreciate input from other folks, since
neither chat2.pl nor Comm.pl works here :-(. In particular, I'm uncertain
how this fits in with the Pty module (which I think also doesn't work
here). Certainly "there's more than one way to do it," but I also think
the distribution should exhibit good habits of library module reuse.

Andy Dougherty doughera@lafcol.lafayette.edu

On Mon, 6 Nov 1995, Tom Christiansen wrote:

> I strongly submit that Chat2 be pulled from the distribution and
> replaced with a shiny new modulated and podified comm library.
> I'm not sure though whether IPC::Comm is the optimal name for it.
> What about IPC::Chat instead?

People are going to be most familiar with the name "expect". Does
IPC::Chat have sufficient "obviousity"? I don't mind it myself, and at
least Chat would ring a bell to anyone who uses UUCP-style "chat
commands".

As for IPC::Comm: "Inter-Proccess Communications::Communication". Could
we have something slightly less redundant, with fewer repetitions, and not
quite as many duplications?

> --tom

--
Kenneth Albanowski (kjahds@kjahds.com, CIS: 70705,126)

On Mon, 6 Nov 1995, Kenneth Albanowski wrote:

> People are going to be most familiar with the name "expect".

People might also expect "expect" to be an interface to "expect." :-)
Let's not use expect just yet.

Andy Dougherty doughera@lafcol.lafayette.edu

On Mon, 6 Nov 1995, Andy Dougherty wrote:

> On Mon, 6 Nov 1995, Kenneth Albanowski wrote:
>
> > People are going to be most familiar with the name "expect".
>
> People might also expect "expect" to be an interface to "expect." :-)
> Let's not use expect just yet.

Good point. IPC::Chat seems reasonable all around.

> Andy Dougherty doughera@lafcol.lafayette.edu

--
Kenneth Albanowski (kjahds@kjahds.com, CIS: 70705,126)

Comm.pl is currently several packages rolled into one because they all
work closely in relation to each other:

- simplified sockets
- PTYs
- expect
- stty/ioctl

Maybe it should be split out a bit. I haven't done that yet, because
in the past it has been convenient to deliver a single catch-all IPC
package.

Should the "open_listen"/"open_port" functions be in their own module,
or part of another, like "Sockets.pm"? "Sockets.pm" is a direct
sockets interface module, and the "Comm.pl" functions are really a
confection layer, so it's not an obvious fit, to me.

Also, given the horrors (see below) I've encountered trying to get PTYs
working on non-BSD/Sun systems, I think that "open3.pm" is still a good
idea, and shouldn't be replaced by "open_proc()" for some time yet.
BTW, I think "open2" should go away, it just causes people to ask for
"open3" a couple days later when they realize they want STDERR on it's
own stream. It should be called "open_pipe" or something.

I've been trying to understand what is required to create a truely
portable PTY package. Pty, Ptty, and everything else I've tried,
including TCL/Expect have certain nasty problems on HP (and also AIX, I
think): anything run in the child shell which opens the tty (i.e.
"stty sane") hangs. My latest hope was to use the TCL/Expect
"exp_spawnv()" C API function to allocate a PTY and, importantly, start
a child process with proper machine-dependent protocol. However, the C
API also causes the hang, although the native TCL/Expect interface does
not. I've included a C program that has the HP hang problem (it spawns a
process, and copies data between it and stdin). If anybody can tell me
what I'm doing wrong, it would help. I haven't gotten a response from
the author, Don Libes.

Newer releases of modulized Comm.pl have been waiting for a portable
PTY interface.

As for the name of the thing, I don't think "chat" is a good name,
because that signifies a particular UUCP style interface which we
aren't even attempting to emulate. "expect" isn't really so bad,
because people will probably recognize what it is trying to do because
it is pretty close to the TCL/Expect idea, but caste in Perl terms.

Leaving aside whether TCL/Expect will form the underlying PTY access,
I'm resistent to creating a full TCL/Expect module, because the
majority of it is redundant with Perl language functionality. I intend
to provide any "Expect" functionality people actually need which isn't
present in the Perl language itself (i.e. "interact()"). Unless
somebody sees a full TCL/Expect module likely, I think it's safe to
use "IPC::Expect".

-Eric

P.S. do we gotta start capitalizing everything? It seems the direction
for all the modules. It also seems a little grandiose to me. If I
remember, there was a stylistic craze in English literature a century
ago where the trendy writers decided they could capitalize any Word
they Thought to be Important. It was Really Annoying to look at!
I suppose it's too late to buck the trend, though...

> From owner-perl5-porters@nicoh.com Mon Nov 6 07:24:19 1995
> Return-Path: <owner-perl5-porters@nicoh.com>
> To: eric.arnold@sun.com, Larry Wall <lwall@netlabs.com>,
> Randal Schwartz <merlyn@stonehenge.com>,
> The Perl Porters Mailing List <perl5-porters@africa.nicoh.com>
> Subject: Comm.pl ---> IPC::Comm.pm
> Reply-To: tchrist@perl.com
> From: Tom Christiansen <tchrist@mox.perl.com>
> Sender: owner-perl5-porters@nicoh.com
> List-Name: perl5-porters
>
> [. I notice only one chat2 bug, NETaa14400. I propose that this email
> be taken as a bug report entitled "upgrade chat2 to use Comm". ]
>
> In working on a FMTEYEWTK on IPC (a new 10x better perlipc.pod, actually),
> I came to the part about IPC::Open[23], and felt bad about explaining the
> buffering issues. (I fixed a couple things in them too, including letting
> you pass \*FOO). In doing so, I just for the heckuvit put together
> corresponding examples using Comm.pl, and it worked like a charm.
>
> require 'Comm.pl';
> $ph = open_proc('cat -n');
> for (1..10) {
> print $ph "a line\n";
> print "got back ", scalar <$ph>;
> }
>
> I strongly submit that Chat2 be pulled from the distribution and
> replaced with a shiny new modulated and podified comm library.
> I'm not sure though whether IPC::Comm is the optimal name for it.
> What about IPC::Chat instead?
>
> It may not work everywhere, but is certainly more versatile than
> chat2, which nearly never works at all, and whose paradigm is
> far too close to eval "asdf" for my tastes.
>
> --tom
>



#include <stdio.h>
#include <sys/types.h>
#include <errno.h>
#include <sys/file.h>
#include <sys/ioctl.h>
#include <signal.h>
#include <sys/time.h>

static struct timeval Timeout;

extern int exp_pid;
extern char *exp_pty_slave_name;
extern char *exp_stty_init;

main(argc,argv)
int argc;
char *argv[];
{
int len=0;
int fd=-1, i=0;
fd_set readfds, writefds, exceptfds;
char *file_name, **p, buf[10000];
int n;

argv++;/* trash own file name */
file_name = *argv;
argv++;

exp_stty_init = "sane";
fd = exp_spawnv(file_name, argv);
printf("tty=%s\n", exp_pty_slave_name );

while(1)
{

FD_ZERO( &readfds );
FD_SET( 0, &readfds );
FD_SET( fd, &readfds );
errno = 0;
if ( !select( FD_SETSIZE, &readfds, &writefds, &exceptfds, 0 ) )
{
printf("select failed, errno=%d\n", errno );
fflush(stdout);
}


if ( FD_ISSET( 0, &readfds ) )
{
do
{
n = read( 0, buf, 1000 ) ;
if ( n < 0 )
{
printf("read 0 failed, dying\n");
exit(1);
}

if ( n == 0 )
break;
for ( i = 0 ; i < n ; i++ )
{
if (buf[i] == '\003' )
{
printf("got ^C, exiting\n");
exit(0);
}
}
buf[n] = 0;
{
/*
printf("gotinput:"); fflush(stdout);
*/
write( fd, buf, n );
}
}
while ( n == 1000 );
}

if ( FD_ISSET( fd, &readfds ) )
{
do
{
n = read( fd, buf, 1000 );
if ( n < 0 )
{
printf("read fd failed, dying\n");
exit(1);
}
if ( n == 0 )
break;
buf[n] = 0;
/*
printf( "read from fd:");
*/
fwrite (buf, n, 1, stdout );
fflush( stdout );
}
while ( n == 1000 );
}

}
}

I've just tried compiling Eric's program with expect 5.6.3 and tcl 7.4 and
can't get it to compile. Expect 5.6.3 wants to find a variable called
"tclRegexpError", which is not defined in Tcl 7.4.

Anyone know what the latest skinny is on Tcl and Expect versions?

Jeff

I compiled and linked Expect 5.6.3 and 5.18 with TCL 7.3.

-Eric

> From okamoto@hpcc123.corp.hp.com Mon Nov 6 11:29:15 1995
> Return-Path: <okamoto@hpcc123.corp.hp.com>
> From: Jeff Okamoto <okamoto@hpcc123.corp.hp.com>
> Subject: Re: exp_spawnv
> To: Eric.Arnold@Corp
> Cc: perl5-porters@nicoh.com
> Mailer: Elm [revision: 70.85]
>
> I've just tried compiling Eric's program with expect 5.6.3 and tcl 7.4 and
> can't get it to compile. Expect 5.6.3 wants to find a variable called
> "tclRegexpError", which is not defined in Tcl 7.4.
>
> Anyone know what the latest skinny is on Tcl and Expect versions?
>
> Jeff
>

>P.S. do we gotta start capitalizing everything? It seems the direction
>for all the modules. It also seems a little grandiose to me.

Well, you should capitalize a module to distinguish it from a pragma:

use Socket;
use strict;

But whether to Capitalize the Functions in the Module is up to you.
Probably not. Theotherissueofcourseiswhetherunderbarsarethere, but
I_think_that_you_know_my_opinion_on_that.

My own code style does this for variables:

$ALL_CAPS global variable, virtual constant
might even *BE* a constant

$Some_Caps Global (static) variable, but might be
in a package other than main.

$no_caps purely local variable

I haven't come up with a good scheme for semidemihemiprivate
variables, like my vs local. I think if you have to use a local()
for something other than merely things like $_ or *FH, that you
should use some caps.

I also don't have so a consistently thought out mechanism for the
function_names a module provides, but usually I eschew upper case
in these entirely, and use _leading_underbars for "internal" functions.
I don't usually bend over backwards to make internal functions
uncallable though (with my()s on closures).

--tom

While I admire Comm.pl's general direction (trying to get chat2
to work), I dislike strongly one very important implementation detail.

In chat2, I worked very hard to keep looks like:

while (something) {
$res = &chat'expect(10,"lookfor","if-found","lookfor2","if-found");
maybe do something here based on $res;
}

from recompiling the regex "lookfor" and "lookfor2" each time through
the loop. Comm.pl has lost this code, and this code *could* be merged
in without too much trouble. (Look at how I created a subroutine and
cached it away.)

However, I think the proper way to do it would be to hand in a bunch
of coderefs:

while (something) {
$res = expect(10,
sub { /lookfor1/ },
sub { /lookfor2/ },
);
do something with $res;
}

and have $res be either the number of the coderef that fired, or
perhaps the non-false value returned from the coderef (I haven't
thought this through).

I wouldn't want people to standardize on a perl4-style interface.
That's why I originally asked for chat2 to be pulled until I finished
hacking out a perl5-style interface, like above. (I really wished
that I had had time to finish this project. Unfortunately, the State
of Oregon and the need to feed my lawyers seems to be swallowing up my
time instead.)

So, I'll state my same concerns about comm.pl that I had about chat2.
The interface for "expect" needs to be *designed* with both efficiency
and ease-of-use in mind. Neither chat2 nor comm meet my minimum
standards.

Name: Randal L. Schwartz / Stonehenge Consulting Services (503)777-0095
Keywords: Perl training, UNIX[tm] consulting, video production, skiing, flying
Email: <merlyn@stonehenge.com> Snail: (Call) PGP-Key: (finger merlyn@ora.com)
Web: <A HREF="http://www.teleport.com/~merlyn/">My Home Page!</A>
Quote: "I'm telling you, if I could have five lines in my .sig, I would!" -- me

> To: lwall@netlabs.com, perl5-porters@africa.nicoh.com, tchrist@perl.com
> Subject: Re: Comm.pl ---> IPC::Comm.pm
> From: "Randal L. Schwartz" <merlyn@teleport.com>
>
....
>
> So, I'll state my same concerns about comm.pl that I had about chat2.
> The interface for "expect" needs to be *designed* with both efficiency
> and ease-of-use in mind. Neither chat2 nor comm meet my minimum
> standards.

Shucks.

When I looked at execution speed, I found that it is going to be pretty
slow no matter what, compared to a regular in-line read (i.e.:
while(<FH>){ do something }
) since there is a lot of overhead in function calls, localizing
variables, and so on. I was unable to gain sufficient speed-up to
justify any fancy stuff.

An in-line read transfers about 1Mb in 1 second on my system.
Un-optimized "Comm::expect()" takes about 27 seconds. When I hard coded
a regexp (i.e. /.+/) I shaved only 2 seconds. I then got rid of any
"local()"s or "my()"s and improved it a little more, another 4
seconds. I fiddled with buffer size and the "select". I commented
out the "wait4/waitpid()" for another 5 seconds. I finally got
it to maybe only 15-20 times slower than an in-line read. I would have
to start throwing away what minimal functionality it now provides to
speed it up in any substantial way.

People won't/shouldn't be using it for huge data anyway. The performance
problems are not noticable when doing the kinds of user-emulation that
Expect is normally asked to do.


I think the ease-of-use has been met. Some tweaking will continue.

As for a perl5 interface, it would be simple to recognize if a code ref
has been passed, and use it, except that I can't get at the $& $' $`
variables after the match. Say you want:

$res = expect(10, sub { /lookfor1/ } );

The caller which deref.s the CODE ref. will not have access to $& $` $'
as this demonstrates:

$ref = sub{/a/} ;

while(<>)
{
$r = &$ref ;
print "yes\n" if $&; # always fails
print "r=$r,&=", $&, "`=", $`, "\n"; # only $r is ever set
}

So, unless somebody can show me how to get around that, you would have
make the user-supplied CODE ref. much too complex in order to allow the
"expect()" function access to $& $` $' which it needs to accumulate
un-used data.

-Eric

> Say you want:

> $res = expect(10, sub { /lookfor1/ } );

>The caller which deref.s the CODE ref. will not have access to $& $` $'
>as this demonstrates:
>
> $ref = sub{/a/} ;
>
> while(<>)
> {
> $r = &$ref ;
> print "yes\n" if $&; # always fails
> print "r=$r,&=", $&, "`=", $`, "\n"; # only $r is ever set
> }

Excellent point. Anyway, it seems to me that sub{/a/} would
just be a high-overhead hackaround for compiled regexps.

--tom

>BTW, I think "open2" should go away, it just causes people to ask for
>"open3" a couple days later when they realize they want STDERR on it's
>own stream. It should be called "open_pipe" or something.

Doesn't seem like a good idea. If you get make everyone say

open3( \*Reader, \*Writer, \*Errer, @cmd )

Then they'll want to say

if ( <Reader> || <Errer> )

(which is horribly ineffectual)

or

Reader->has_data || Errer->has_data

or somesuch. In any event, they'll end up having to deal with
noneblocking I/O, and as we know, select() doesn't work well with
stdio. This means they'll have to use sysread() instead of their
normal <Reader> stuff, and that's just way much hassle.

So I think that the simple stuff like

use FileHandle;
use IPC::Open2;
$pid = open2( \*Reader, \*Writer, "cat -u -n" );
Writer->autoflush(); # default here, actually
print Writer "stuff\n";
$got = <Reader>;

shouldn't get broken and convuluted into multiplexed I/O
without a darned good reason, cause we'll have to through
out stdio.

Larry once asked me to write a FH->getline() that used sysread()
until $buff ended in $RS. Now I remember why. :-(

--tom

In <9511070045.AA08275@animus.Corp.Sun.COM>
On Mon, 6 Nov 95 16:45:51 PST
Eric Arnold <Eric.Arnold@Corp.Sun.COM> writes:
>> To: lwall@netlabs.com, perl5-porters@africa.nicoh.com, tchrist@perl.com
>> Subject: Re: Comm.pl ---> IPC::Comm.pm
>> From: "Randal L. Schwartz" <merlyn@teleport.com>
>>
>....
>
>As for a perl5 interface, it would be simple to recognize if a code ref
>has been passed, and use it, except that I can't get at the $& $' $`
>variables after the match. Say you want:
>
> $res = expect(10, sub { /lookfor1/ } );
>
>The caller which deref.s the CODE ref. will not have access to $& $` $'
>as this demonstrates:
>
> $ref = sub{/a/} ;
>
> while(<>)
> {
> $r = &$ref ;
> print "yes\n" if $&; # always fails
> print "r=$r,&=", $&, "`=", $`, "\n"; # only $r is ever set
> }
>
>So, unless somebody can show me how to get around that, you would have
>make the user-supplied CODE ref. much too complex in order to allow the
>"expect()" function access to $& $` $' which it needs to accumulate
>un-used data.

why not have the sub return an array when it matches and an empty list
otherwise. That way the user can decide what he/she wants from the sub.
expect could the return this list.

eg

@res = expect(10,
sub { /lookfor1/ && return (1) }, # simple value
sub { /lookfor2/ && return ($',$&,$`) } # list of values
);


Regards,
Graham.

--
Graham Barr <bodg@tiuk.ti.com>
Texas Instruments Ltd.

> From: Tom Christiansen <tchrist@mox.perl.com>
>
> > Say you want:
>
> > $res = expect(10, sub { /lookfor1/ } );
>
> >The caller which deref.s the CODE ref. will not have access to $& $` $'
> >as this demonstrates:
> >
> > $ref = sub{/a/} ;
> >
> > while(<>)
> > {
> > $r = &$ref ;
> > print "yes\n" if $&; # always fails
> > print "r=$r,&=", $&, "`=", $`, "\n"; # only $r is ever set
> > }
>
> Excellent point. Anyway, it seems to me that sub{/a/} would
> just be a high-overhead hackaround for compiled regexps.

All we need to do is to get Larry to log this as a Sev 2 bug and he'll hack
in refs to compiled regexps one afternoon (presumably after a pumpkin lunch).

:-)

Tim.