Mailing List Archive

On Tue, Nov 19, 2013 at 6:34 PM, John Napiorkowski <jjn1056@yahoo.com>wrote:

>
> This looks familiar, but I thought we fixed this... what version of
> Catalyst are you using here?
>

I just updated:


$ perl -MCatalyst -le 'print Catalyst->VERSION'
5.90051

I can debug more in the morning -- anything specific I should look at?

It's pretty easy to reproduce:

package Zip::Controller::Root;
...

use IO::Uncompress::Gunzip;
use IO::Compress::Gzip qw/ gzip $GzipError /;

sub uncompress : Local {
my ( $self, $c ) = @_;

my $uncompressed = "This is some text that can be compressed.\n" x 5;
my $compressed;
gzip( \$uncompressed, \$compressed ) || die $GzipError;

$c->res->body( IO::Uncompress::Gunzip->new( \$compressed ) );

return;
}



$ script/zip_test.pl /uncompress

-s on unopened filehandle GEN2 at
/home/bill/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst.pm
line 1948, <DATA> line 1000.
[warn] Serving filehandle without a content-length
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.


--
Bill Moseley
moseley@hank.org

On 20/11/2013 3:01 PM, Bill Moseley wrote:
>
> I just updated:
>
> $ perl -MCatalyst -le 'print Catalyst->VERSION'
> 5.90051
>
> I can debug more in the morning -- anything specific I should look at?
I think the stat operators have this problem on any type of in memory
filehandle. That is pretty easy to reproduce.
To me anyhow this wouldn't seem to make much sense to me to test the
length of a yet uncompressed filehandle, even if it did just work somehow.

For my money I would do the following:

my $body = IO::Uncompress::Gunzip->new( \$gzipped_data); #Assuming scalar
my $length = $body->getHeaderInfo->{ISIZE};
$c->res->content_length( $length );
$c->res->body( $body );

So basically getting the length of the uncompressed data from the handle
by the object method. So now Catalyst won't try to set the size because
you already have. Oh, and the content type and well of course :)



>
> It's pretty easy to reproduce:
>
> package Zip::Controller::Root;
> ...
>
> use IO::Uncompress::Gunzip;
> use IO::Compress::Gzip qw/ gzip $GzipError /;
>
> sub uncompress : Local {
> my ( $self, $c ) = @_;
>
> my $uncompressed = "This is some text that can be
> compressed.\n" x 5;
> my $compressed;
> gzip( \$uncompressed, \$compressed ) || die $GzipError;
>
> $c->res->body( IO::Uncompress::Gunzip->new( \$compressed ) );
> return;
> }
>
>
>
> $ script/zip_test.pl <http://zip_test.pl> /uncompress
>
> -s on unopened filehandle GEN2 at
> /home/bill/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst.pm
> line 1948, <DATA> line 1000.
> [warn] Serving filehandle without a content-length
> This is some text that can be compressed.
> This is some text that can be compressed.
> This is some text that can be compressed.
> This is some text that can be compressed.
> This is some text that can be compressed.
>
>
> --
> Bill Moseley
> moseley@hank.org <mailto:moseley@hank.org>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On Wed, Nov 20, 2013 at 4:08 AM, neil.lunn <neil@mylunn.id.au> wrote:

> my $length = $body->getHeaderInfo


Well, that's helpful. Thanks. Completely missed that in the docs.


--
Bill Moseley
moseley@hank.org

On Wed, Nov 20, 2013 at 7:37 AM, Bill Moseley <moseley@hank.org> wrote:

>
> On Wed, Nov 20, 2013 at 4:08 AM, neil.lunn <neil@mylunn.id.au> wrote:
>
>> my $length = $body->getHeaderInfo
>
>
> Well, that's helpful. Thanks. Completely missed that in the docs.
>

Well, except for the actual files I have no ISIZE:

$VAR1 = {
'Time' => 0,
'Flags' => 0,
'TextFlag' => 0,
'MethodID' => 8,
'ExtraField' => [],
'CommentFlag' => 0,
'Type' => 'rfc1952',
'NameFlag' => 0,
'ExtraFlags' => 0,
'HeaderCRC' => undef,
'isMinimalHeader' => 0,
'MethodName' => 'Deflated',
'ExtraFlag' => 0,
'HeaderLength' => 10,
'ExtraFieldRaw' => undef,
'Comment' => undef,
'OsName' => 'Unix',
'FingerprintLength' => 2,
'HeaderCRCFlag' => 0,
'OsID' => 3,
'TrailerLength' => 8,
'Name' => undef,
'Header' => ''
};

And even if I gzip a file (e.g. $ gzip foo.txt) then the resulting
foo.txt.gz has 'ISIZE' => 0,



--
Bill Moseley
moseley@hank.org

On 21/11/2013 4:08 AM, Bill Moseley wrote:
>
> On Wed, Nov 20, 2013 at 7:37 AM, Bill Moseley <moseley@hank.org
> <mailto:moseley@hank.org>> wrote:
>
>
> On Wed, Nov 20, 2013 at 4:08 AM, neil.lunn <neil@mylunn.id.au
> <mailto:neil@mylunn.id.au>> wrote:
>
> my $length = $body->getHeaderInfo
>
>
> Well, that's helpful. Thanks. Completely missed that in the docs.
>
>
> Well, except for the actual files I have no ISIZE:

Don't know. I would suspect you have a version issue somewhere. But not
a Catalyst thing. From the Gzip RFC:

<snip>

2.3.1.2. Compliance

A compliant compressor must produce files with correct ID1,
ID2, CM, CRC32, and ISIZE, but may set all the other fields in
the fixed-length part of the header to default values (255 for
OS, 0 for all others). The compressor must set all reserved
bits to zero.

</snip>

>
> $VAR1 = {
> 'Time' => 0,
> 'Flags' => 0,
> 'TextFlag' => 0,
> 'MethodID' => 8,
> 'ExtraField' => [],
> 'CommentFlag' => 0,
> 'Type' => 'rfc1952',
> 'NameFlag' => 0,
> 'ExtraFlags' => 0,
> 'HeaderCRC' => undef,
> 'isMinimalHeader' => 0,
> 'MethodName' => 'Deflated',
> 'ExtraFlag' => 0,
> 'HeaderLength' => 10,
> 'ExtraFieldRaw' => undef,
> 'Comment' => undef,
> 'OsName' => 'Unix',
> 'FingerprintLength' => 2,
> 'HeaderCRCFlag' => 0,
> 'OsID' => 3,
> 'TrailerLength' => 8,
> 'Name' => undef,
> 'Header' => ''
> };
>
> And even if I gzip a file (e.g. $ gzip foo.txt) then the resulting
> foo.txt.gz has 'ISIZE' => 0,
>
>
>
> --
> Bill Moseley
> moseley@hank.org <mailto:moseley@hank.org>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On Wed, Nov 20, 2013 at 3:45 PM, neil.lunn <neil@mylunn.id.au> wrote:

> 2.3.1.2. Compliance
>
>
> A compliant compressor must produce files with correct ID1,
> ID2, CM, CRC32, and ISIZE, but may set all the other fields in
> the fixed-length part of the header to default values (255 for
> OS, 0 for all others). The compressor must set all reserved
> bits to zero.
>
> </snip>
>

Seems noncompliance may be rampant.

Anyway, sounds like Catalyst isn't quite supporting this kind of file
handle as expected. John, is there anything you would want me to try?




$ perl -MIO::Uncompress::Gunzip -le 'use Data::Dumper; print Dumper
+IO::Uncompress::Gunzip->new( "Catalyst-Runtime-5.90051.tar.gz"
)->getHeaderInfo'
$VAR1 = {
'Time' => 1383843952,
'Flags' => 8,
'TextFlag' => 0,
'MethodID' => 8,
'ExtraField' => [],
'CommentFlag' => 0,
'Type' => 'rfc1952',
'NameFlag' => 1,
'ExtraFlags' => 2,
'HeaderCRC' => undef,
'isMinimalHeader' => 0,
'MethodName' => 'Deflated',
'ExtraFlag' => 0,
'HeaderLength' => 39,
'ExtraFieldRaw' => undef,
'Comment' => undef,
'OsName' => 'Unix',
'FingerprintLength' => 2,
'HeaderCRCFlag' => 0,
'OsID' => 3,
'TrailerLength' => 8,
'Name' => 'Catalyst-Runtime-5.90051.tar',
'Header' => p�{RCatalyst-Runtime-5.90051.tar'
};



--
Bill Moseley
moseley@hank.org

On 21/11/2013 11:48 AM, Bill Moseley wrote:
>
> Seems noncompliance may be rampant.
>
> Anyway, sounds like Catalyst isn't quite supporting this kind of file
> handle as expected. John, is there anything you would want me to try?
>

Hi Bill. Back to my original response, trying to get the size of this
handle (or any other type of in memory handle) with a file stat operator
will not work. What catalyst is trying to do:

$size = -s $response->body;

Because it reasonably expects you have provided a handle that can read
and most importantly you have not provided a content length. Being the
earlier conditional.

Thinking this through, if the file was on disk then -s would not return
the uncompressed size. Which is what you want.

As for that not working for this type of handle ( aslo see something
like IO::Scalar ), I can't see why Catalyst should be expected to do
that for you. Which is why there is the provision to provide a
content_length and avioding this fallback condition.

Your issue as I see it, is that something is borked on your setup with
the Gzip implementation that is stopping you from getting the
uncompressed size from the content. Therefore I suggest you start
looking there.

Again I cannot see how or why Catalyst could or should be expected to
work this out for you, and the best method is to set the correct
content_length once you have sorted out the Gzip issue. Or failing that
do you have some other way in your implementation to know the
content_length when uncompressed.

As below just works, which is as near as I can see to what you are
basically doing.


package Gzip::Web::Controller::Root;
use Moose;
use namespace::autoclean;

BEGIN { extends 'Catalyst::Controller' }

use IO::Compress::Gzip qw/gzip $GzipError/;
use IO::Uncompress::Gunzip;
use Data::Dumper;

__PACKAGE__->config(namespace => '');

sub index :Path :Args(0) {
my ( $self, $c ) = @_;

my $data = "123456890ABCDEFGHIGQWERRTYYUIO";
my ($comp, $body);

gzip(\$data, \$comp) || die $GzipError;

$c->res->content_type('text/plain');

if ( $c->req->header('accept-encoding') =~ /gzip/ ) {
$c->log->debug( 'Sending compressed' );
$c->res->content_encoding('gzip');
$body = $comp;
} else {
$c->log->debug( 'Sending uncompressed' );
$body = IO::Uncompress::Gunzip->new( \$comp );
$c->res->content_length( $body->getHeaderInfo->{ISIZE} );
$c->log->debug( Dumper( $body->getHeaderInfo ) );
}

$c->res->body( $body );

}



>
>
> $ perl -MIO::Uncompress::Gunzip -le 'use Data::Dumper; print Dumper
> +IO::Uncompress::Gunzip->new( "Catalyst-Runtime-5.90051.tar.gz"
> )->getHeaderInfo'
> $VAR1 = {
> 'Time' => 1383843952,
> 'Flags' => 8,
> 'TextFlag' => 0,
> 'MethodID' => 8,
> 'ExtraField' => [],
> 'CommentFlag' => 0,
> 'Type' => 'rfc1952',
> 'NameFlag' => 1,
> 'ExtraFlags' => 2,
> 'HeaderCRC' => undef,
> 'isMinimalHeader' => 0,
> 'MethodName' => 'Deflated',
> 'ExtraFlag' => 0,
> 'HeaderLength' => 39,
> 'ExtraFieldRaw' => undef,
> 'Comment' => undef,
> 'OsName' => 'Unix',
> 'FingerprintLength' => 2,
> 'HeaderCRCFlag' => 0,
> 'OsID' => 3,
> 'TrailerLength' => 8,
> 'Name' => 'Catalyst-Runtime-5.90051.tar',
> 'Header' => p?{RCatalyst-Runtime-5.90051.tar'
> };
>
>
> --
> Bill Moseley
> moseley@hank.org <mailto:moseley@hank.org>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On 21/11/2013 1:04 PM, neil.lunn wrote:
> On 21/11/2013 11:48 AM, Bill Moseley wrote:
>>
>> $ perl -MIO::Uncompress::Gunzip -le 'use Data::Dumper; print Dumper
>> +IO::Uncompress::Gunzip->new( "Catalyst-Runtime-5.90051.tar.gz"
>> )->getHeaderInfo'
Oh, and a test case outside of catalyst, noting the -s operator fail:


echo "1234567890ABCDEFG" | gzip -c >x.gz

#!/usr/bin/env perl
use Modern::Perl;

use File::Slurp;
use IO::Uncompress::Gunzip;
use Data::Dumper;

my $z = read_file "x.gz";
my $fh = IO::Uncompress::Gunzip->new( \$z );

my $size = -s $fh;

print Dumper( $fh->getHeaderInfo );
print "Size: ", $fh->getHeaderInfo->{ISIZE}, "\n";
print $fh->getline();
$fh->close();

-s on unopened filehandle GEN0 at test.pl line 11.
$VAR1 = {
'NameFlag' => 0,
'TrailerLength' => 8,
'Header' => '??R',
'CRC32' => 3714309751,
'ISIZE' => 18,
'ExtraFlag' => 0,
'Flags' => 0,
'Comment' => undef,
'isMinimalHeader' => 0,
'ExtraField' => [],
'HeaderLength' => 10,
'OsID' => 3,
'MethodID' => 8,
'MethodName' => 'Deflated',
'HeaderCRC' => undef,
'TextFlag' => 0,
'Name' => undef,
'CommentFlag' => 0,
'HeaderCRCFlag' => 0,
'Type' => 'rfc1952',
'FingerprintLength' => 2,
'Time' => 1384950479,
'ExtraFlags' => 0,
'ExtraFieldRaw' => undef,
'OsName' => 'Unix'
};
Size: 18
1234567890ABCDEFG

>> $VAR1 = {
>> 'Time' => 1383843952,
>> 'Flags' => 8,
>> 'TextFlag' => 0,
>> 'MethodID' => 8,
>> 'ExtraField' => [],
>> 'CommentFlag' => 0,
>> 'Type' => 'rfc1952',
>> 'NameFlag' => 1,
>> 'ExtraFlags' => 2,
>> 'HeaderCRC' => undef,
>> 'isMinimalHeader' => 0,
>> 'MethodName' => 'Deflated',
>> 'ExtraFlag' => 0,
>> 'HeaderLength' => 39,
>> 'ExtraFieldRaw' => undef,
>> 'Comment' => undef,
>> 'OsName' => 'Unix',
>> 'FingerprintLength' => 2,
>> 'HeaderCRCFlag' => 0,
>> 'OsID' => 3,
>> 'TrailerLength' => 8,
>> 'Name' => 'Catalyst-Runtime-5.90051.tar',
>> 'Header' => p?{RCatalyst-Runtime-5.90051.tar'
>> };
>>
>>
>> --
>> Bill Moseley
>> moseley@hank.org <mailto:moseley@hank.org>
>>
>>
>> _______________________________________________
>> List:Catalyst@lists.scsys.co.uk
>> Listinfo:http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive:http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>> Dev site:http://dev.catalyst.perl.org/
>
>
>
> ------------------------------------------------------------------------
> <http://www.avast.com/>
>
> This email is free from viruses and malware because avast! Antivirus
> <http://www.avast.com/> protection is active.
>
>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On 21/11/2013 11:48 AM, Bill Moseley wrote:
>>>
>>> $ perl -MIO::Uncompress::Gunzip -le 'use Data::Dumper; print Dumper
>>> +IO::Uncompress::Gunzip->new( "Catalyst-Runtime-5.90051.tar.gz"
>>> )->getHeaderInfo'

Actually with more reading on this getting the uncompressed size
reliably can be a real pain, and thus is not in the getHeaderInfo over
certain size lmits.

This approach may work for you is the compressed data is actually in a
scalar and not too large. And not too small. YMMV.

my $z = read_file "product.json.gz";

my $io = IO::Scalar->new( \$z );
$io->seek( -4, 2 );
$io->read( my $buf, 4);

my $uncompressed_size = unpack( 'V', $buf );

>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn <neil@mylunn.id.au> wrote:

>
> This approach may work for you is the compressed data is actually in a
> scalar and not too large. And not too small. YMMV.
>
> my $z = read_file "product.json.gz";
>
> my $io = IO::Scalar->new( \$z );
> $io->seek( -4, 2 );
> $io->read( my $buf, 4);
>
> my $uncompressed_size = unpack( 'V', $buf );
>


This indeed does work in my tests. Thanks for all the help, Neil. I
really appreciate the time you spent on this.



--
Bill Moseley
moseley@hank.org

On 22/11/2013 12:33 AM, Bill Moseley wrote:
>
>
>
> On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn <neil@mylunn.id.au
> <mailto:neil@mylunn.id.au>> wrote:
>
>
> This approach may work for you is the compressed data is actually
> in a scalar and not too large. And not too small. YMMV.
>
> my $z = read_file "product.json.gz";
>
> my $io = IO::Scalar->new( \$z );
> $io->seek( -4, 2 );
> $io->read( my $buf, 4);
>
> my $uncompressed_size = unpack( 'V', $buf );
>
>
>
> This indeed does work in my tests. Thanks for all the help, Neil.
> I really appreciate the time you spent on this.
>
No prob. Part of my thing on list reponses is that there is a seed for
others to learn from what was discussed.
I do wonder if there is a sane way to test for an in memory file handle
and give a more appropriate warning, but that is for further discussion.

>
>
> --
> Bill Moseley
> moseley@hank.org <mailto:moseley@hank.org>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

We should probably document what it means by $c->response->body can accept a file handle.  Like plack specifies the interface it must deal with.  Anyone up to that?

This FH in body thing is something I've been thinking about a lot lately, in regards to when Catalyst is running under an event loop for example, should Catalyst stream the Filehandle a non blocking manner.  But it would be great to document this.  Also, I am thinking setting the default size when its missing could really belong to middleware.  Plack has some default middleware that does this:

https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Middleware/ContentLength.pm


the means it works looks like this

https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Util.pm#L68


There's a few things in Catalyst.pm that could profitable just use Plack::Middleware counterparts.

just a thought.

johnnap



On Thursday, November 21, 2013 9:20 AM, neil.lunn <neil@mylunn.id.au> wrote:

On 22/11/2013 12:33 AM, Bill Moseley wrote:


>
>
>
>
>On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn <neil@mylunn.id.au> wrote:
>
>
>>
This approach may work for you is the compressed data is actually in a scalar and not too large. And not too small. YMMV.
>>
>>my $z = read_file "product.json.gz";
>>
>>my $io = IO::Scalar->new( \$z );
>>$io->seek( -4, 2 );
>>$io->read( my $buf, 4);
>>
>>my $uncompressed_size = unpack( 'V', $buf );
>
>
>
>
>This indeed does work in my tests.   Thanks for all the help, Neil.   I really appreciate the time you spent on this.
>
>
No prob. Part of my thing on list reponses is that there is a seed for others to learn from what was discussed.
I do wonder if there is a sane way to test for an in memory file
handle and give a more appropriate warning, but that is for further
discussion.



>
>
>
--
>Bill Moseley
>moseley@hank.org
>
>
>_______________________________________________
List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/


________________________________

This email is free from viruses and malware because avast! Antivirus protection is active.


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

On 26/11/2013 12:54 PM, John Napiorkowski wrote:
> We should probably document what it means by $c->response->body can
> accept a file handle. Like plack specifies the interface it must deal
> with. Anyone up to that?

Something along these lines maybe?

Catalyst can accept a file handle type of object as the response body in
the sense that the handle reasonably has a "read" method that will
return content. In such a case and where no content length has been
already set in the response headers catalyst will make a reasonable
attempt to determine the size of the file. Depending on the
implementation of your file handle object setting the content length may
fail. If it is at all possible for you to determine the content length
of your file handle, it is recomended that you set the content length in
the response headers yourself, which will be respected and sent by
catalyst in the response.
>
> This FH in body thing is something I've been thinking about a lot
> lately, in regards to when Catalyst is running under an event loop for
> example, should Catalyst stream the Filehandle a non blocking manner.
> But it would be great to document this. Also, I am thinking setting
> the default size when its missing could really belong to middleware.
> Plack has some default middleware that does this:
>
> https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Middleware/ContentLength.pm

Which interestingly does not take a too different approach to what the
Catalyst code is attempting. Also noting that in a case such as Bill
presented it too would fail, which is only to be expected.

Noting such IO magic modules like IO::Uncompress::* and IO::Scalar etc
and speaking of cat skinning, another approach could be subclassing
these modues and overloading the file test operators to do something
sane when used, and thus return a "correct" response when the -s test is
used. Or try and convince said authors to have these classes do this
natively.

>
> the means it works looks like this
>
> https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Util.pm#L68
>
> There's a few things in Catalyst.pm that could profitable just use
> Plack::Middleware counterparts.
>
> just a thought.
>
> johnnap
>
>
> On Thursday, November 21, 2013 9:20 AM, neil.lunn <neil@mylunn.id.au>
> wrote:
> On 22/11/2013 12:33 AM, Bill Moseley wrote:
>>
>>
>>
>> On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn <neil@mylunn.id.au
>> <mailto:neil@mylunn.id.au>> wrote:
>>
>>
>> This approach may work for you is the compressed data is actually
>> in a scalar and not too large. And not too small. YMMV.
>>
>> my $z = read_file "product.json.gz";
>>
>> my $io = IO::Scalar->new( \$z );
>> $io->seek( -4, 2 );
>> $io->read( my $buf, 4);
>>
>> my $uncompressed_size = unpack( 'V', $buf );
>>
>>
>>
>> This indeed does work in my tests. Thanks for all the help, Neil. I
>> really appreciate the time you spent on this.
>>
> No prob. Part of my thing on list reponses is that there is a seed for
> others to learn from what was discussed.
> I do wonder if there is a sane way to test for an in memory file
> handle and give a more appropriate warning, but that is for further
> discussion.
>
>
>>
>>
>> --
>> Bill Moseley
>> moseley@hank.org <mailto:moseley@hank.org>
>>
>>
>> _______________________________________________ List:
>> Catalyst@lists.scsys.co.uk <mailto:Catalyst@lists.scsys.co.uk>
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive:
>> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site:
>> http://dev.catalyst.perl.org/
>>
>
>
>
> ------------------------------------------------------------------------
> <http://www.avast.com/>
> This email is free from viruses and malware because avast! Antivirus
> <http://www.avast.com/> protection is active.
>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk <mailto:Catalyst@lists.scsys.co.uk>
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/



---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

Awesome, send me a pull request :)

https://github.com/perl-catalyst/catalyst-runtime




On Tuesday, November 26, 2013 1:24 AM, neil.lunn <neil@mylunn.id.au> wrote:

On 26/11/2013 12:54 PM, John Napiorkowski wrote:

We should probably document what it means by $c->response->body can accept a file handle.  Like plack specifies the interface it must deal with.  Anyone up to that?
Something along these lines maybe?

Catalyst can accept a file handle type of object as the response
body in the sense that the handle reasonably has a "read" method
that will return content. In such a case and where no content length
has been already set in the response headers catalyst will make a
reasonable attempt to determine the size of the file. Depending on
the implementation of your file handle object setting the content
length may fail. If it is at all possible for you to determine the
content length of your file handle, it is recomended that you set
the content length in the response headers yourself, which will be
respected and sent by catalyst in the response.


>
>This FH in body thing is something I've been thinking about a lot lately, in regards to when Catalyst is running under an event loop for example, should Catalyst stream the Filehandle a non blocking manner.  But it would be great to document this.  Also, I am thinking setting the default size when its missing could really belong to middleware.  Plack has some default middleware that does this:
>
>
>https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Middleware/ContentLength.pm
>
Which interestingly does not take a too different approach to what
the Catalyst code is attempting. Also noting that in a case such as
Bill presented it too would fail, which is only to be expected.

Noting such IO magic modules like IO::Uncompress::* and IO::Scalar
etc and speaking of cat skinning, another approach could be
subclassing these modues and overloading the file test operators to
do something sane when used, and thus return a "correct" response
when the -s test is used. Or try and convince said authors to have
these classes do this natively.



>
>the means it works looks like this
>
>
>https://metacpan.org/source/MIYAGAWA/Plack-1.0030/lib/Plack/Util.pm#L68
>
>
>
>There's a few things in Catalyst.pm that could profitable just use Plack::Middleware counterparts.
>
>
>just a thought.
>
>
>johnnap
>
>
>
>On Thursday, November 21, 2013 9:20 AM, neil.lunn <neil@mylunn.id.au> wrote:
>
>On 22/11/2013 12:33 AM, Bill Moseley wrote:
>
>
>>
>>
>>
>>
>>On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn <neil@mylunn.id.au> wrote:
>>
>>
>>>
This approach may work for you is the compressed data is actually in a scalar and not too large. And not too small. YMMV.
>>>
>>>my $z = read_file "product.json.gz";
>>>
>>>my $io = IO::Scalar->new( \$z );
>>>$io->seek( -4, 2 );
>>>$io->read( my $buf, 4);
>>>
>>>my $uncompressed_size = unpack( 'V',
$buf );
>>
>>
>>
>>
>>This indeed does work in my tests.   Thanks for all the help, Neil.   I really appreciate the time you spent on this.
>>
>>
No prob. Part of my thing on list reponses is that there is a seed for others to learn from what was discussed.
>I do wonder if there is a sane way to test for an in
memory file handle and give a more appropriate
warning, but that is for further discussion.
>
>
>
>
>>
>>
>>
--
>>Bill Moseley
>>moseley@hank.org
>>
>>
>>_______________________________________________
List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
>
>
>>________________________________
>
> This email is free from viruses and malware because avast! Antivirus protection is active.
>
>
>
>_______________________________________________
>List: Catalyst@lists.scsys.co.uk
>Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>Dev site: http://dev.catalyst.perl.org/
>
>
>
>
>
>
>
>
>_______________________________________________
List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/



________________________________

This email is free from viruses and malware because avast! Antivirus protection is active.

* Craig Chant <craig@homeloanpartnership.com> [2013-11-26 10:20]:
> # create an IO::File for Catalyst
> use IO::File;
> my $iof = IO::File->new;
>
> # open XLS
> $iof->open(\$xls, "r");

open my $iof, '<', \$xls;

--
*AUTOLOAD=*_;sub _{s/..([^:]*)$/()[print$1,(",$\/"," ")[defined wantarray]]/e;$_}
&Just->another->Perl->hack;
#Aristotle Pagaltzis // <http://plasmasturm.org/>

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

On 27/11/2013 4:28 AM, John Napiorkowski wrote:
> Awesome, send me a pull request :)
>
> https://github.com/perl-catalyst/catalyst-runtime
John.
The pull is there along with another doc patch I noticed when playing
around with downstream PSGI invocations. I think the change is sane as
it worked for me in a trivial test.

What I mentioned regarding subclassing ( or rather what I tested was
more of a Moose delegation ) for such a class as IO::Compress::Gunzip
and applying an overload seems rather trivial, though perhaps something
people may come accross. Perhaps a doc patch to Catalyst::Manual or an
Advent entry just for documentation sake.

Also passing $c->res->body is something I have used on a few occasions
as something providing a "write" for streaming responses (and possibly
non blocking). So keen to share any thoughts re doing this non-blocking
for "read" on a future release.

Should we really be documenting some of the newer catalyst features in
Cookbook? Or at least putting recipes in Advent . Event loop things,
response from PSGI and how about Pocket::IO seem relevant.


---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

On Wednesday, November 27, 2013 8:03 AM, neil.lunn <neil@mylunn.id.au> wrote:

On 27/11/2013 4:28 AM, John Napiorkowski wrote:

Awesome, send me a pull request :)
>
>
>https://github.com/perl-catalyst/catalyst-runtime
>
John.
The pull is there along with another doc patch I noticed when
    playing around with downstream PSGI invocations. I think the change
    is sane as it worked for me in a trivial test.

What I mentioned regarding subclassing ( or rather what I tested was
    more of a Moose delegation ) for such a class as
    IO::Compress::Gunzip and applying an overload seems rather trivial,
    though perhaps something people may come accross. Perhaps a doc
    patch to Catalyst::Manual or an Advent entry just for documentation
    sake.

Also passing $c->res->body is something I have used on a few
    occasions as something providing a "write" for streaming responses
    (and possibly non blocking). So keen to share any thoughts re doing
    this non-blocking for "read" on a future release. 

There's a few approaches, you can also access the $response->write_fh which returns the psgi $writer (see the PSGI docs for delayed and streaming responses. 


Should we really be documenting some of the newer catalyst features
    in Cookbook? Or at least putting recipes in Advent . Event loop
    things, response from PSGI and how about Pocket::IO seem relevant.

Advent will have a bunch of articles related to new features, and some stuff around web sockets. nonblocking.

If someone wants to port any of the web sockets examples to pocket.io or similar, for running on older browsers, that would be great.  If you know about pocket.io feel free to take a look over at the catalyst a-synch example repository on github : https://github.com/jjn1056/Perl-Catalyst-AsyncExample

Thanks!

john



________________________________

   This email is free from viruses and malware because avast! Antivirus protection is active.  

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/