Mailing List Archive

force the output encoding to utf-8 in page templates
Hi all!

I've a Zope2 application with Postgresql. All is ok and utf-8 but page
templates. This means that non utf-8 data is sent to the database, and
the database complain with:

*Error Type: DataError*
*Error Value: invalid byte sequence for encoding "UTF8": 0xe82527 HINT:
This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by "client_encoding".

*This does not happen if I manually switch the browser encoding (Firefox
-> View -> character encoding -> Unicode (UTF-8)) and submit a form.

So my question is: how do I tell to page templates to output utf-8
instead of iso8859-15? The output_encoding property does anything.

Obviously, changing default-zpublisher-encoding to utf-8 works, but
other legacy applications will have problems on characted encoding.

Any idea?
**
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: force the output encoding to utf-8 in page templates [ In reply to ]
Il 11/11/2011 12:43, Yuri ha scritto:
> Hi all!
>
> I've a Zope2 application with Postgresql. All is ok and utf-8 but
> page templates. This means that non utf-8 data is sent to the
> database, and the database complain with:
>
> *Error Type: DataError*
> *Error Value: invalid byte sequence for encoding "UTF8": 0xe82527
> HINT: This error can also happen if the byte sequence does not match
> the encoding expected by the server, which is controlled by
> "client_encoding".
>
> *This does not happen if I manually switch the browser encoding
> (Firefox -> View -> character encoding -> Unicode (UTF-8)) and submit
> a form.
>
> So my question is: how do I tell to page templates to output utf-8
> instead of iso8859-15? The output_encoding property does anything.
>
> Obviously, changing default-zpublisher-encoding to utf-8 works, but
> other legacy applications will have problems on characted encoding.
>
> Any idea?
> **


I've solved in page templates code:

if not response.headers.has_key('content-type'):
response.setHeader('content-type', self.content_type +
';charset=' + self.output_encoding) <- here add the utf-8, the
publisher will find this filled and avoid to put the default.

on line 315 of ZopePageTemplate.py

Is there a better way? Thanks!
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: force the output encoding to utf-8 in page templates [ In reply to ]
On Fri, 11 Nov 2011 12:56:30 +0100, Yuri <yurj@alfa.it> wrote:
> Il 11/11/2011 12:43, Yuri ha scritto:
>> Hi all!
>>
>> I've a Zope2 application with Postgresql. All is ok and utf-8 but
>> page templates. This means that non utf-8 data is sent to the
>> database, and the database complain with:
>>
>> *Error Type: DataError*
>> *Error Value: invalid byte sequence for encoding "UTF8": 0xe82527
>> HINT: This error can also happen if the byte sequence does not match
>> the encoding expected by the server, which is controlled by
>> "client_encoding".
>>
>> *This does not happen if I manually switch the browser encoding
>> (Firefox -> View -> character encoding -> Unicode (UTF-8)) and submit
>> a form.
>>
>> So my question is: how do I tell to page templates to output utf-8
>> instead of iso8859-15? The output_encoding property does anything.
>>
>> Obviously, changing default-zpublisher-encoding to utf-8 works, but
>> other legacy applications will have problems on characted encoding.
>>
>> Any idea?
>> **
>
>
> I've solved in page templates code:
>
> if not response.headers.has_key('content-type'):
> response.setHeader('content-type', self.content_type +
> ';charset=' + self.output_encoding) <- here add the utf-8, the
> publisher will find this filled and avoid to put the default.
>
> on line 315 of ZopePageTemplate.py
>
> Is there a better way? Thanks!

Look for setting the default publisher encoding in the zope.conf
filesystem file and restart. :)

-Morten
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: force the output encoding to utf-8 in page templates [ In reply to ]
Il 11/11/2011 13:40, morten@nidelven-it.no ha scritto:
> On Fri, 11 Nov 2011 12:56:30 +0100, Yuri<yurj@alfa.it> wrote:
>> Il 11/11/2011 12:43, Yuri ha scritto:
>>> Hi all!
>>>
>>> I've a Zope2 application with Postgresql. All is ok and utf-8 but
>>> page templates. This means that non utf-8 data is sent to the
>>> database, and the database complain with:
>>>
>>> *Error Type: DataError*
>>> *Error Value: invalid byte sequence for encoding "UTF8": 0xe82527
>>> HINT: This error can also happen if the byte sequence does not match
>>> the encoding expected by the server, which is controlled by
>>> "client_encoding".
>>>
>>> *This does not happen if I manually switch the browser encoding
>>> (Firefox -> View -> character encoding -> Unicode (UTF-8)) and submit
>>> a form.
>>>
>>> So my question is: how do I tell to page templates to output utf-8
>>> instead of iso8859-15? The output_encoding property does anything.
>>>
>>> Obviously, changing default-zpublisher-encoding to utf-8 works, but
>>> other legacy applications will have problems on characted encoding.
>>>
>>> Any idea?
>>> **
>>
>> I've solved in page templates code:
>>
>> if not response.headers.has_key('content-type'):
>> response.setHeader('content-type', self.content_type +
>> ';charset=' + self.output_encoding)<- here add the utf-8, the
>> publisher will find this filled and avoid to put the default.
>>
>> on line 315 of ZopePageTemplate.py
>>
>> Is there a better way? Thanks!
> Look for setting the default publisher encoding in the zope.conf
> filesystem file and restart. :)
>
> -Morten

Even forcing utf-8 in the content-type header, I still have this error
on some web browser. For example, mozilla firefox 3.6.18 on windows
gives this error:

Error Type: DataError
Error Value: invalid byte sequence for encoding "UTF8": 0xe82075 HINT:
This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by "client_encoding".

It's sad there's no standard or no way to be compliant.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: force the output encoding to utf-8 in page templates [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

You should be able to set the encoding of the response by

<tal:def define="dummy python:
request.RESPONSE.setHeader('content-type', 'text/html; charset=utf-8')" />

within your page template.

- -aj

Yuri wrote:
> Hi all!
>
> I've a Zope2 application with Postgresql. All is ok and utf-8 but
> page templates. This means that non utf-8 data is sent to the
> database, and the database complain with:
>
> *Error Type: DataError* *Error Value: invalid byte sequence for
> encoding "UTF8": 0xe82527 HINT: This error can also happen if the
> byte sequence does not match the encoding expected by the server,
> which is controlled by "client_encoding".
>
> *This does not happen if I manually switch the browser encoding
> (Firefox -> View -> character encoding -> Unicode (UTF-8)) and submit
> a form.
>
> So my question is: how do I tell to page templates to output utf-8
> instead of iso8859-15? The output_encoding property does anything.
>
> Obviously, changing default-zpublisher-encoding to utf-8 works, but
> other legacy applications will have problems on characted encoding.
>
> Any idea? ** _______________________________________________ Zope
> maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope ** No cross posts or
> HTML encoding! ** (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )

- --
ZOPYX Limited | zopyx group
Charlottenstr. 37/1 | The full-service network for Zope & Plone
D-72070 Tübingen | Produce & Publish
www.zopyx.com | www.produce-and-publish.com
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJOwhBcAAoJEADcfz7u4AZj5hkLvjkql56PD0CzUKp1JZkLta4n
WirPCXII9j8tFptsVrMs8OuRg60TUDwF7MCGptI/gf5MAxsHbCgKH0NDYr1PHLQx
5BWtGXsQ/5t6Lblu7xJywuOWuBan82k6i6LDmvrW5OyFupNpMduhaLPad1Uy5s3L
now6drULzDkML4hvBGbsw63Ufh/7IIa4S71lf6pxtcS/O0zEXUrEJ4BNf4hIezOW
Usl7v98ZK4GppN3/rJgP/NQLQ23elkxysLWpK4YmpByAQ/7Lx/rdkqpqMA6DwgCJ
qDQiWvij6yHQwHmTbHIXd/NkexH4SU5UviQZhWdSG8a+ZqygYEd/sGXbHUhtTMRc
V0bE2v6zDhY1jpvk1VFthug8IbmNmOYeRiVuhb4mMMlXOnQ3BEaIrxe8ultJHVKM
OAwhWI7XHX/wL/TTmoZc1v4jhEkcD6sLE6TlqXd5XsNf2D84IA4hOSnajnuWg8nQ
zmJTTn7CZNo+p+13moXKqXaWErXFT/A=
=1Maw
-----END PGP SIGNATURE-----
Re: force the output encoding to utf-8 in page templates [ In reply to ]
Il 15/11/2011 08:10, Andreas Jung ha scritto:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> You should be able to set the encoding of the response by
>
> <tal:def define="dummy python:
> request.RESPONSE.setHeader('content-type', 'text/html; charset=utf-8')" />
>
> within your page template.

<tal:def tal:define="dummy
python:request.RESPONSE.setHeader('content-type', 'text/html;;
charset=utf-8')" /> is the correct one.

I've to test it now :)
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )