Mailing List Archive

UnicodeDecodeError
Hi!

I've an Zope application, basically postgres + zpt.

The postgres db has been upgraded to 8.5, the data inside is utf-8
(dumped and checkd it), the pg adapeter is the latest, and the "*Z
Psycopg 2 Database Connection" is configured to use utf-8.* Zope is
2.12.20 + latest security fix.

When I try to do:

<span tal:replace="result/?column" /> (result came from an ZSQL method)
I get an UnicodeDecodeError:

Traceback (innermost last):
Module ZPublisher.Publish, line 127, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 47, in call_object
Module Shared.DC.Scripts.Bindings, line 324, in __call__
Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec
Module Products.PageTemplates.ZopePageTemplate, line 335, in _exec
Module Products.PageTemplates.ZopePageTemplate, line 432, in pt_render
Module Products.PageTemplates.PageTemplate, line 80, in pt_render
Module zope.pagetemplate.pagetemplate, line 113, in pt_render
Module zope.tal.talinterpreter, line 271, in __call__
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 888, in do_useMacro
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 946, in do_defineSlot
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 533, in do_optTag_tal
Module zope.tal.talinterpreter, line 518, in do_optTag
Module zope.tal.talinterpreter, line 513, in no_tag
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 821, in do_loop_tal
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 821, in do_loop_tal
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 852, in do_condition
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 376, in do_startEndTag
Module zope.tal.talinterpreter, line 405, in do_startTag
Module zope.tal.talinterpreter, line 502, in attrAction_tal
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 38:
ordinal not in range(128)

if I do:

<span tal:replace="python:str(result['column']).decode('utf-8')" /> it
works. (I use str because I've also dates in the record, otherwise I
could not decode being it not a string)

I've several zpt to fix, and I would like it to work as it should. My
analisys is that result['column'] is seen as a plain string with \xx
characters inside, and decode just transform it in something that Zope
recognize as utf-8 when it prints it.

Anyway, any idea?
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: UnicodeDecodeError [ In reply to ]
Hi Yuri,

does this error occur with all browsers? I once had a similar issue
that was caused only by some browsers because they do not send a
HTTP_ACCEPT_CHARSET header.

Maybe the following bug reports helps you in solving the issue:
https://bugs.launchpad.net/zope2/+bug/160968

I'm using a workaround similar to that in the bug report in my
environment (Zope 2.13.13 and a mixture of utf-8 and iso8859).
Without the workaround the issue does still occur for me.

Greetings,
Stefan

Am 03.04.2012 09:40, schrieb Yuri:
> Hi!
>
> I've an Zope application, basically postgres + zpt.
>
> The postgres db has been upgraded to 8.5, the data inside is utf-8
> (dumped and checkd it), the pg adapeter is the latest, and the "*Z
> Psycopg 2 Database Connection" is configured to use utf-8.* Zope is
> 2.12.20 + latest security fix.
>
> When I try to do:
>
> <span tal:replace="result/?column" /> (result came from an ZSQL method)
> I get an UnicodeDecodeError:
>
> Traceback (innermost last):
> Module ZPublisher.Publish, line 127, in publish
> Module ZPublisher.mapply, line 77, in mapply
> Module ZPublisher.Publish, line 47, in call_object
> Module Shared.DC.Scripts.Bindings, line 324, in __call__
> Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec
> Module Products.PageTemplates.ZopePageTemplate, line 335, in _exec
> Module Products.PageTemplates.ZopePageTemplate, line 432, in pt_render
> Module Products.PageTemplates.PageTemplate, line 80, in pt_render
> Module zope.pagetemplate.pagetemplate, line 113, in pt_render
> Module zope.tal.talinterpreter, line 271, in __call__
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 888, in do_useMacro
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 946, in do_defineSlot
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 533, in do_optTag_tal
> Module zope.tal.talinterpreter, line 518, in do_optTag
> Module zope.tal.talinterpreter, line 513, in no_tag
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 821, in do_loop_tal
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 821, in do_loop_tal
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 852, in do_condition
> Module zope.tal.talinterpreter, line 343, in interpret
> Module zope.tal.talinterpreter, line 376, in do_startEndTag
> Module zope.tal.talinterpreter, line 405, in do_startTag
> Module zope.tal.talinterpreter, line 502, in attrAction_tal
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 38:
> ordinal not in range(128)
>
> if I do:
>
> <span tal:replace="python:str(result['column']).decode('utf-8')" /> it
> works. (I use str because I've also dates in the record, otherwise I
> could not decode being it not a string)
>
> I've several zpt to fix, and I would like it to work as it should. My
> analisys is that result['column'] is seen as a plain string with \xx
> characters inside, and decode just transform it in something that Zope
> recognize as utf-8 when it prints it.
>
> Anyway, any idea?
> _______________________________________________
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )

--
Stefan Loidl Phone: +49 89 35831 8882
Leibniz-Rechenzentrum Fax: +49 89 35831 8682
Boltzmannstr. 1 mailto:loidl@lrz.de
85748 Garching http://www.lrz.de
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )