Mailing List Archive

utf8 on c->res-write or c->res->print
Hello,

I'm trying to put some utf8 strings as a part of Catalyst response as
follows:

$c->res->print ( ... )
and
$c->res->write( ... )

line by line.

However it dies with

Wide character in syswrite at ..... IO/Handle.pm line 474

Any ideas how can utf8 be written to response?

E.g. set binmode on output handle. Cannot find how.
Using Catalyst Runtime 5.90042.

Thanks.

HtH, Sergey Dmitriev
*"All that is necessary for evil to triumph is for good men to do nothing".
-- Edmund Burke*
Re: utf8 on c->res-write or c->res->print [ In reply to ]
use utf8;
my $fh = $c->res->write_fh;
my $test = "\x{41f}\x{440}\x{43e}\x{431}\x{430}";
utf8::encode($test);
$fh->write($test);

On 20 February 2014 17:45, Sergey Dmitriev <sergey.programmer@gmail.com> wrote:
> Hello,
>
> I'm trying to put some utf8 strings as a part of Catalyst response as
> follows:
>
> $c->res->print ( ... )
> and
> $c->res->write( ... )
>
> line by line.
>
> However it dies with
>
> Wide character in syswrite at ..... IO/Handle.pm line 474
>
> Any ideas how can utf8 be written to response?
>
> E.g. set binmode on output handle. Cannot find how.
> Using Catalyst Runtime 5.90042.
>
> Thanks.
>
> HtH, Sergey Dmitriev
> "All that is necessary for evil to triumph is for good men to do nothing".
> -- Edmund Burke
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>



--
//wbr, Dmitry L.

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: utf8 on c->res-write or c->res->print [ In reply to ]
On Feb 20, 2014, at 2:19 PM, Dmitry L. <dim0xff@gmail.com> wrote:

> use utf8;

You shouldn’t need this unless you have actual utf8 characters (rather than escapes) in your source code.

> my $fh = $c->res->write_fh;
> my $test = "\x{41f}\x{440}\x{43e}\x{431}\x{430}";
> utf8::encode($test);

You probably actually want Encode::encode(‘UTF-8’, $test);

> $fh->write($test);
>
> On 20 February 2014 17:45, Sergey Dmitriev <sergey.programmer@gmail.com> wrote:
>> Hello,
>>
>> I'm trying to put some utf8 strings as a part of Catalyst response as
>> follows:
>>

Right, you can’t do that :)

If you try to use the raw write interfaces you have to send bytes not characters (and so need to encode your utf8 character strings).

>> $c->res->print ( ... )
>> and
>> $c->res->write( ... )
>>
>> line by line.
>>
>> However it dies with
>>
>> Wide character in syswrite at ..... IO/Handle.pm line 474
>>
>> Any ideas how can utf8 be written to response?
>>
>> E.g. set binmode on output handle. Cannot find how.
>> Using Catalyst Runtime 5.90042.
>>

This is not what you want to do.

The handle is already binary - just you’re not writing binary to it, you’re writing characters (and perl does not know how to translate those characters into bytes).

Cheers
Tom


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: utf8 on c->res-write or c->res->print [ In reply to ]
>
> Thank you, it works! by using Encode::encode("UTF-8", 'my text' ).
>

And just for reference, if anyone will use it, HTTP headers should be set
BEFORE calling write_fh.
e.g.
$c->res->header( 'Cache-Control' => 'public, max-age=600' );
my $fh = $c->res->write_fh; # psgi writer, not filehandle

Otherwise headers will not be set.

HtH, Sergey Dmitriev
*"All that is necessary for evil to triumph is for good men to do nothing".
-- Edmund Burke*


2014-02-20 18:53 GMT+04:00 Tomas Doran <bobtfish@bobtfish.net>:

>
> On Feb 20, 2014, at 2:19 PM, Dmitry L. <dim0xff@gmail.com> wrote:
>
> > use utf8;
>
> You shouldn't need this unless you have actual utf8 characters (rather
> than escapes) in your source code.
>
> > my $fh = $c->res->write_fh;
> > my $test = "\x{41f}\x{440}\x{43e}\x{431}\x{430}";
> > utf8::encode($test);
>
> You probably actually want Encode::encode('UTF-8', $test);
>
> > $fh->write($test);
> >
> > On 20 February 2014 17:45, Sergey Dmitriev <sergey.programmer@gmail.com>
> wrote:
> >> Hello,
> >>
> >> I'm trying to put some utf8 strings as a part of Catalyst response as
> >> follows:
> >>
>
> Right, you can't do that :)
>
> If you try to use the raw write interfaces you have to send bytes not
> characters (and so need to encode your utf8 character strings).
>
> >> $c->res->print ( ... )
> >> and
> >> $c->res->write( ... )
> >>
> >> line by line.
> >>
> >> However it dies with
> >>
> >> Wide character in syswrite at ..... IO/Handle.pm line 474
> >>
> >> Any ideas how can utf8 be written to response?
> >>
> >> E.g. set binmode on output handle. Cannot find how.
> >> Using Catalyst Runtime 5.90042.
> >>
>
> This is not what you want to do.
>
> The handle is already binary - just you're not writing binary to it,
> you're writing characters (and perl does not know how to translate those
> characters into bytes).
>
> Cheers
> Tom
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
Re: utf8 on c->res-write or c->res->print [ In reply to ]
On Friday, February 21, 2014 7:25 AM, Sergey Dmitriev <sergey.programmer@gmail.com> wrote:

Thank you, it works! by using Encode::encode("UTF-8", 'my text' ).
>

And just for reference, if anyone will use it, HTTP headers should be set BEFORE calling write_fh.
e.g.
    $c->res->header( 'Cache-Control'  => 'public, max-age=600' );
    my $fh = $c->res->write_fh; # psgi writer, not filehandle


Otherwise headers will not be set.


===

Correct, once you call ->write_fh OR ->write on Response, we must finalize the headers. This is because the streaming interface can't stream a HTTP body without first sending HTTP headers.  By using ->write/_fh you have chosen to bypass part of the Catalyst finalization process and talk more directly to the requesting client.

In newer versions of catalyst trying to set headers after requesting a write handler will generate an error.

--John

===


HtH, Sergey Dmitriev
"All that is necessary for evil to triumph is for good men to do nothing". -- Edmund Burke



2014-02-20 18:53 GMT+04:00 Tomas Doran <bobtfish@bobtfish.net>:


>On Feb 20, 2014, at 2:19 PM, Dmitry L. <dim0xff@gmail.com> wrote:
>
>>    use utf8;
>
>You shouldn’t need this unless you have actual utf8 characters (rather than escapes) in your source code.
>
>
>>    my $fh = $c->res->write_fh;
>>    my $test = "\x{41f}\x{440}\x{43e}\x{431}\x{430}";
>>    utf8::encode($test);
>
>You probably actually want Encode::encode(‘UTF-8’, $test);
>
>
>>    $fh->write($test);
>>
>> On 20 February 2014 17:45, Sergey Dmitriev <sergey.programmer@gmail.com> wrote:
>>> Hello,
>>>
>>> I'm trying to put some utf8 strings as a part of Catalyst response as
>>> follows:
>>>
>
>Right, you can’t do that :)
>
>If you try to use the raw write interfaces you have to send bytes not characters (and so need to encode your utf8 character strings).
>
>
>>> $c->res->print ( ... )
>>> and
>>> $c->res->write( ... )
>>>
>>> line by line.
>>>
>>> However it dies with
>>>
>>>   Wide character in syswrite at ..... IO/Handle.pm line 474
>>>
>>> Any ideas how can utf8 be written to response?
>>>
>>> E.g. set binmode on output handle. Cannot find how.
>>> Using Catalyst Runtime 5.90042.
>>>
>
>This is not what you want to do.
>
>The handle is already binary - just you’re not writing binary to it, you’re writing characters (and perl does not know how to translate those characters into bytes).
>
>Cheers
>Tom
>
>
>
>_______________________________________________
>List: Catalyst@lists.scsys.co.uk
>Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>Dev site: http://dev.catalyst.perl.org/
>


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/