Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Zope>
  3. Users
charset problems (utf-8)
ramosh at gmail
Feb 2, 2012, 10:06 AM
Post #1 of 8 (4552 views)
Permalink
Yellow,

I've noticed charset problems displaying HTML pages.

Let's say I want to display a page using a DTML Method like this:

<html>
<head>
...
</head>

<body>
á Á à À ç
</body>
</html>

This page is ok and I can see the portuguese letters fine.

The problem starts when I do this:

<html>
<head>
...
</head>

<body>
á Á à À ç <br>
<dtml-var sometextfield>
</body>
</html>

The first line of characters shows some strange characters and the
sometextfield shows up fine even if it has the same portuguese
characters.


sometextfield comes from a mySQL server using utf-8 as charset and collation.

zope.conf has the following:

rest-input-encoding utf-8
rest-output-encoding utf-8

The browsers also have UTF-8 as the encoding (safari, firefox and
chrome) all show me the same problem.

I also noticed some strange behavior...
If I create "á Á à À ç" as a property field using ustring and call it
as <dtml-var someproperty> also gets fine in the browser.

Anyone seen this??? Double encoding maybe???


TIA

--
Hugo Ramos - IT Project Manager
E: ramosh@gmail.com
W: www.hugoramos.eu

"Nobody expects the Spanish Inquisition!" - Monty Python's

Para visualizar este email em 3D, bata com uma bola de snooker na
testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: charset problems (utf-8) [ In reply to ]
ramosh at gmail
Feb 6, 2012, 4:38 AM
Post #2 of 8 (4550 views)
Permalink
I guess no one has charset problems...



On Thu, Feb 2, 2012 at 6:06 PM, Hugo Ramos <ramosh@gmail.com> wrote:
> Yellow,
>
> I've noticed charset problems displaying HTML pages.
>
> Let's say I want to display a page using a DTML Method like this:
>
> <html>
> <head>
> ...
> </head>
>
> <body>
> á Á à À ç
> </body>
> </html>
>
> This page is ok and I can see the portuguese letters fine.
>
> The problem starts when I do this:
>
> <html>
> <head>
> ...
> </head>
>
> <body>
> á Á à À ç <br>
> <dtml-var sometextfield>
> </body>
> </html>
>
> The first line of characters shows some strange characters and the
> sometextfield shows up fine even if it has the same portuguese
> characters.
>
>
> sometextfield comes from a mySQL server using utf-8 as charset and collation.
>
> zope.conf has the following:
>
> rest-input-encoding utf-8
> rest-output-encoding utf-8
>
> The browsers also have UTF-8 as the encoding (safari, firefox and
> chrome) all show me the same problem.
>
> I also noticed some strange behavior...
> If I create "á Á à À ç" as a property field using ustring and call it
> as <dtml-var someproperty> also gets fine in the browser.
>
> Anyone seen this??? Double encoding maybe???
>
>
> TIA
>
> --
> Hugo Ramos - IT Project Manager
> E: ramosh@gmail.com
> W: www.hugoramos.eu
>
> "Nobody expects the Spanish Inquisition!" - Monty Python's
>
> Para visualizar este email em 3D, bata com uma bola de snooker na
> testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.



--
Hugo Ramos - IT Project Manager
E: ramosh@gmail.com
W: www.hugoramos.eu

"Nobody expects the Spanish Inquisition!" - Monty Python's

Para visualizar este email em 3D, bata com uma bola de snooker na
testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: charset problems (utf-8) [ In reply to ]
objectvalues at gmail
Feb 6, 2012, 7:24 AM
Post #3 of 8 (4518 views)
Permalink
Hi,

You have to make sure that everything has the same encoding (strings, text
values). I recomend you to convert everything to unicode and then put
something like below as a first line in your page:

<span
tal:replace="python:request.RESPONSE.setHeader('content-type','text/html;charset=utf-8')"
/>

and in the HEAD

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

HTH,
Dragos


On Mon, Feb 6, 2012 at 2:38 PM, Hugo Ramos <ramosh@gmail.com> wrote:

> I guess no one has charset problems...
>
>
>
> On Thu, Feb 2, 2012 at 6:06 PM, Hugo Ramos <ramosh@gmail.com> wrote:
> > Yellow,
> >
> > I've noticed charset problems displaying HTML pages.
> >
> > Let's say I want to display a page using a DTML Method like this:
> >
> > <html>
> > <head>
> > ...
> > </head>
> >
> > <body>
> > á Á à À ç
> > </body>
> > </html>
> >
> > This page is ok and I can see the portuguese letters fine.
> >
> > The problem starts when I do this:
> >
> > <html>
> > <head>
> > ...
> > </head>
> >
> > <body>
> > á Á à À ç <br>
> > <dtml-var sometextfield>
> > </body>
> > </html>
> >
> > The first line of characters shows some strange characters and the
> > sometextfield shows up fine even if it has the same portuguese
> > characters.
> >
> >
> > sometextfield comes from a mySQL server using utf-8 as charset and
> collation.
> >
> > zope.conf has the following:
> >
> > rest-input-encoding utf-8
> > rest-output-encoding utf-8
> >
> > The browsers also have UTF-8 as the encoding (safari, firefox and
> > chrome) all show me the same problem.
> >
> > I also noticed some strange behavior...
> > If I create "á Á à À ç" as a property field using ustring and call it
> > as <dtml-var someproperty> also gets fine in the browser.
> >
> > Anyone seen this??? Double encoding maybe???
> >
> >
> > TIA
> >
> > --
> > Hugo Ramos - IT Project Manager
> > E: ramosh@gmail.com
> > W: www.hugoramos.eu
> >
> > "Nobody expects the Spanish Inquisition!" - Monty Python's
> >
> > Para visualizar este email em 3D, bata com uma bola de snooker na
> > testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
>
>
>
> --
> Hugo Ramos - IT Project Manager
> E: ramosh@gmail.com
> W: www.hugoramos.eu
>
> "Nobody expects the Spanish Inquisition!" - Monty Python's
>
> Para visualizar este email em 3D, bata com uma bola de snooker na
> testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
> _______________________________________________
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )
>



--
Dragos Chirila
objectValues@gmail.com
(+4) 0722 395375
Re: charset problems (utf-8) [ In reply to ]
ramosh at gmail
Feb 6, 2012, 7:58 AM
Post #4 of 8 (4511 views)
Permalink
The correct prefix in the HTML header (<meta http-equiv="Content-Type"
content="text/html; charset=utf-8"> is not needed because it affects
the browser behavior only and all my 3 browsers are set to UTF-8 by
default.

I've checked and I have some templates encoded as utf-8 and others as
iso-8859-1. This happens because some of the templates (mainly the
ones encoded as iso-8859-1) come from old data.fs that I've upgraded
and maintained over the years...
But this doesn't seem to be the problem because templates encoded as
utf-8 or iso-8859-1 all have the same problem.

Any more ideas?


BTW, any1 knows about some script I can use to encode everything in my
data.fs to utf-8? That would be very useful!




On Mon, Feb 6, 2012 at 2:16 PM, Thierry Florac <thierry.florac@onf.fr> wrote:
>
> Hi,
>
> As a french, I had many charset problems at first because of our
> accentuated characters...
>
> But your problems are quite strange if everything is encoded in UTF-8.
>
> Do you have the correct prefix in your HTML template as well as the
> correct HTTP headers ?
> Can you check that your template is really encoded in UTF-8 ??
>
> Regards,
> Thierry
>
>
> Le Mon, 06 Feb 2012 12:38:22 +0000,
>  Hugo Ramos <ramosh@gmail.com> a écrit:
>
>> I guess no one has charset problems...
>>
>>
>>
>> On Thu, Feb 2, 2012 at 6:06 PM, Hugo Ramos <ramosh@gmail.com> wrote:
>> > Yellow,
>> >
>> > I've noticed charset problems displaying HTML pages.
>> >
>> > Let's say I want to display a page using a DTML Method like this:
>> >
>> > <html>
>> > <head>
>> > ...
>> > </head>
>> >
>> > <body>
>> > á Á à À ç
>> > </body>
>> > </html>
>> >
>> > This page is ok and I can see the portuguese letters fine.
>> >
>> > The problem starts when I do this:
>> >
>> > <html>
>> > <head>
>> > ...
>> > </head>
>> >
>> > <body>
>> > á Á à À ç <br>
>> > <dtml-var sometextfield>
>> > </body>
>> > </html>
>> >
>> > The first line of characters shows some strange characters and the
>> > sometextfield shows up fine even if it has the same portuguese
>> > characters.
>> >
>> >
>> > sometextfield comes from a mySQL server using utf-8 as charset and
>> > collation.
>> >
>> > zope.conf has the following:
>> >
>> > rest-input-encoding utf-8
>> > rest-output-encoding utf-8
>> >
>> > The browsers also have UTF-8 as the encoding (safari, firefox and
>> > chrome) all show me the same problem.
>> >
>> > I also noticed some strange behavior...
>> > If I create "á Á à À ç" as a property field using ustring and call
>> > it as <dtml-var someproperty> also gets fine in the browser.
>> >
>> > Anyone seen this??? Double encoding maybe???
>> >
>> >
>> > TIA
>> >
>> > --
>> > Hugo Ramos - IT Project Manager
>> > E: ramosh@gmail.com
>> > W: www.hugoramos.eu
>> >
>> > "Nobody expects the Spanish Inquisition!" - Monty Python's
>> >
>> > Para visualizar este email em 3D, bata com uma bola de snooker na
>> > testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.



--
Hugo Ramos - IT Project Manager
E: ramosh@gmail.com
W: www.hugoramos.eu

"Nobody expects the Spanish Inquisition!" - Monty Python's

Para visualizar este email em 3D, bata com uma bola de snooker na
testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: charset problems (utf-8) [ In reply to ]
ramosh at gmail
Feb 6, 2012, 7:59 AM
Post #5 of 8 (4517 views)
Permalink
Is there any tool I can use to convert all objects in my data.fs to utf-8?




On Mon, Feb 6, 2012 at 3:24 PM, Dragos Chirila <objectvalues@gmail.com> wrote:
> Hi,
>
> You have to make sure that everything has the same encoding (strings, text
> values). I recomend you to convert everything to unicode and then put
> something like below as a first line in your page:
>
> <span
> tal:replace="python:request.RESPONSE.setHeader('content-type','text/html;charset=utf-8')"
> />
>
> and in the HEAD
>
> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
>
> HTH,
> Dragos
>
>
> On Mon, Feb 6, 2012 at 2:38 PM, Hugo Ramos <ramosh@gmail.com> wrote:
>>
>> I guess no one has charset problems...
>>
>>
>>
>> On Thu, Feb 2, 2012 at 6:06 PM, Hugo Ramos <ramosh@gmail.com> wrote:
>> > Yellow,
>> >
>> > I've noticed charset problems displaying HTML pages.
>> >
>> > Let's say I want to display a page using a DTML Method like this:
>> >
>> > <html>
>> > <head>
>> > ...
>> > </head>
>> >
>> > <body>
>> > á Á à À ç
>> > </body>
>> > </html>
>> >
>> > This page is ok and I can see the portuguese letters fine.
>> >
>> > The problem starts when I do this:
>> >
>> > <html>
>> > <head>
>> > ...
>> > </head>
>> >
>> > <body>
>> > á Á à À ç <br>
>> > <dtml-var sometextfield>
>> > </body>
>> > </html>
>> >
>> > The first line of characters shows some strange characters and the
>> > sometextfield shows up fine even if it has the same portuguese
>> > characters.
>> >
>> >
>> > sometextfield comes from a mySQL server using utf-8 as charset and
>> > collation.
>> >
>> > zope.conf has the following:
>> >
>> > rest-input-encoding utf-8
>> > rest-output-encoding utf-8
>> >
>> > The browsers also have UTF-8 as the encoding (safari, firefox and
>> > chrome) all show me the same problem.
>> >
>> > I also noticed some strange behavior...
>> > If I create "á Á à À ç" as a property field using ustring and call it
>> > as <dtml-var someproperty> also gets fine in the browser.
>> >
>> > Anyone seen this??? Double encoding maybe???
>> >
>> >
>> > TIA
>> >
>> > --
>> > Hugo Ramos - IT Project Manager
>> > E: ramosh@gmail.com
>> > W: www.hugoramos.eu
>> >
>> > "Nobody expects the Spanish Inquisition!" - Monty Python's
>> >
>> > Para visualizar este email em 3D, bata com uma bola de snooker na
>> > testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
>>
>>
>>
>> --
>> Hugo Ramos - IT Project Manager
>> E: ramosh@gmail.com
>> W: www.hugoramos.eu
>>
>> "Nobody expects the Spanish Inquisition!" - Monty Python's
>>
>> Para visualizar este email em 3D, bata com uma bola de snooker na
>> testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
>> _______________________________________________
>> Zope maillist  -  Zope@zope.org
>> https://mail.zope.org/mailman/listinfo/zope
>> **   No cross posts or HTML encoding!  **
>> (Related lists -
>>  https://mail.zope.org/mailman/listinfo/zope-announce
>>  https://mail.zope.org/mailman/listinfo/zope-dev )
>
>
>
>
> --
> Dragos Chirila
> objectValues@gmail.com
> (+4) 0722 395375
>



--
Hugo Ramos - IT Project Manager
E: ramosh@gmail.com
W: www.hugoramos.eu

"Nobody expects the Spanish Inquisition!" - Monty Python's

Para visualizar este email em 3D, bata com uma bola de snooker na
testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: charset problems (utf-8) [ In reply to ]
fernando at cmartins
Feb 6, 2012, 4:11 PM
Post #6 of 8 (4513 views)
Permalink
On 02/06/2012 04:59 PM, Hugo Ramos wrote:
> Is there any tool I can use to convert all objects in my data.fs to utf-8?

I had unicode related problems which usually come from mixing different
encodings. In my case pretty much all my data also comes from a
relational database. Also, not all versions of Zope support unicode
properly for all types of objects. I had problems trying to upgrade my
data.fs between zope versions due to unicode issues. I don't recall all
the situations I came through and I can't help you much more here.

So, I guess your issue is that your page templates are combining data
from other zope objects which are in different encoding. I doubt that
you can convert all your objects to the same encoding as a general rule
because that will be dependent on the specific object implementation.
Even if you convert the data part of the objects, the code might still
not work properly. But I don't know which type of objects you are
talking about. Maybe one approach would be to export your data.fs as a
whole or in parts using XML format and then try to make sure its
encoding is utf-8.

Just vague considerations and a long shot, sorry if it doesn't help.

Regards,
Fernando

_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: charset problems (utf-8) [ In reply to ]
yourpadre at gmail
Feb 9, 2012, 1:13 PM
Post #7 of 8 (4516 views)
Permalink
2012/2/6 Hugo Ramos <ramosh@gmail.com>



> But this doesn't seem to be the problem because templates encoded as
> utf-8 or iso-8859-1 all have the same problem.
>
> Any more ideas?
>
>
> are you played witth *default-zpublisher-encoding* option in zope.conf?

________________________________________
Lo bueno de vivir un dia mas
es saber que nos queda un dia menos de vida
Re: charset problems (utf-8) [ In reply to ]
ramosh at gmail
Feb 10, 2012, 1:36 AM
Post #8 of 8 (4486 views)
Permalink
Hi Miguel,

Yes, as I wrote in my first email, all encoding options in zope.conf
are set to utf-8...



On Thu, Feb 9, 2012 at 9:13 PM, Miguel Beltran R. <yourpadre@gmail.com> wrote:
>
>
> 2012/2/6 Hugo Ramos <ramosh@gmail.com>
>
>
>>
>> But this doesn't seem to be the problem because templates encoded as
>> utf-8 or iso-8859-1 all have the same problem.
>>
>> Any more ideas?
>>
>>
> are you played witth default-zpublisher-encoding option in zope.conf?
>
> ________________________________________
> Lo bueno de vivir un dia mas
> es saber que nos queda un dia menos de vida



--
Hugo Ramos - IT Project Manager
E: ramosh@gmail.com
W: www.hugoramos.eu

"Nobody expects the Spanish Inquisition!" - Monty Python's

Para visualizar este email em 3D, bata com uma bola de snooker na
testa e fixe o ecrã durante 5 minutos a uma distância de 20cm.
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal