Mailing List Archive

Hi Yianni!

On 11/08/2013 1:41 PM, Yianni Vidalis wrote:
> I have started translating mythweb and I've come across something that I think is a bug. It concerns the Categories translations.
>
> I've created Greek.cat, Greek.lang and I've added the line:
>
> 'Greek' => array('ÅëëçíéêÜ', 'el_GR', 'el_GR'),
>
> in the Translate.php file as per the instructions at
http://www.mythtv.org/wiki/Translation#Translate_MythWeb
>

I am wondering if your problem might not be related to the fact that
there are no character set declaration (either UTF-8 or, I belive,
ISO-8859-7 (those el_GR entries should specify the character set like
some of the other entries you can see in the same file).

> I'm enclosing a small abstract from the Greek.cat file:
>
> "Crime_Mystery"
> ÁóôõíïìéêÞ_Ìõóôçñßïõ
> \b(?:ÁóôõíïìéêÞ|ìõóôçñ|crim|myster) (I've kept the English expressions as well)
>
> To avoid the language barrier, I'll try to simplify:
>
> "Crime_Mystery"
> GRCrime_GRMystery
> \b(?:grpolice|grmyst|crim|myster)
>
>
>
> The way it should work if I understand correctly is: The program searches the db for the value of the Category column in "Program" table. It compares it to the regular expressions in the third line, i.e. words that start with grpolice, or grmyst, or crim or myster and then shows "GRCrime_GRMystery" in mythweb.

Yep, that's how it should work...

>
> It doesn't work like this, though. The genre color changes successfully only if the Category column contains GRCrime_GRMystery, and it's case sensitive.

It's like it's not processing the right line...

The first thing I would look at is the character set used (does your
editor tell you). I would try to use UTF-8.

The second thing I could think of, did you edit that file solely on
Linux/BSD/etc... PC or could it have been edited on a PC that would have
added CR (ie Windows) that might also confuse it...

>
> Should I report this as a bug, or am I misunderstanding something?
>

Let's see if we can eliminate the possibility of a character set or line
termination issue first...

Thank you!

Nicolas
_______________________________________________
Mythtv-translators mailing list
Mythtv-translators@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-translators

> I am wondering if your problem might not be related to the fact that
> there are no character set declaration (either UTF-8 or, I belive,
> ISO-8859-7 (those el_GR entries should specify the character set like
> some of the other entries you can see in the same file).
>

Ok, thanks for the tip, I was following "German" as an example and not "English".

Now I changed it to

        'Greek'         => array('ÅëëçíéêÜ',           'el_GR.UTF-8', 'el_GR.UTF-8'),

i.e. as reported by locale.



> It's like it's not processing the right line...
>
> The first thing I would look at is the character set used (does your
> editor tell you). I would try to use UTF-8.
>
> The second thing I could think of, did you edit that file solely on
> Linux/BSD/etc... PC or could it have been edited on a PC that would have
> added CR (ie Windows) that might also confuse it...
>

The file is saved as UTF-8, as it was edited in kwrite (openSuSE 12.3).


>>
>> Should I report this as a bug, or am I misunderstanding something?
>>
>
> Let's see if we can eliminate the possibility of a character set or line
> termination issue first...

Sorry, the problem remains. It has to be the exact same case word or phrase
of the second line (the translated one). The regular expressions are ignored.



>
> Thank you!
>
> Nicolas

Thank you.
_______________________________________________
Mythtv-translators mailing list
Mythtv-translators@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-translators

Hi Yianni!

> Sorry, the problem remains. It has to be the exact same case word or phrase
> of the second line (the translated one). The regular expressions are ignored.

Can you please send me that file, I'll try to track down the problem...

Thank you!

Nicolas
_______________________________________________
Mythtv-translators mailing list
Mythtv-translators@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-translators

> Can you please send me that file, I'll try to track down the problem...
>
> Thank you!
>
> Nicolas


Good hunting and many thanks,

Yianni.

Hi Yianni!

Yianni Vidalis wrote:

> Good hunting and many thanks,

You're welcome...

OK, I have not completed my investigation and I am not a PHP programmer
but I think I have a pretty good idea of what is going on...

> The way it should work if I understand correctly is: The program searches the db for the value of the Category column in "Program" table. It compares it to the regular expressions in the third line, i.e. words that start with grpolice, or grmyst, or crim or myster and then shows "GRCrime_GRMystery" in mythweb.

Yep, that's what's supposed to happen...

> It doesn't work like this, though. The genre color changes successfully only if the Category column contains GRCrime_GRMystery, and it's case sensitive.

That too, surprisingly, is supposed to happen but it is supposed to be
case insensitive...

Even before evaluating the regex on the program category it checks if
its matches the translation (I assume that's done to avoid the overhead
of evaluating the regex).

It's supposed to be case insensitive though... The reason why it isn't
is the same why the pattern matching of the regex doesn't work... It's
not able to properly convert the program category and the translation to
lower case to see if they match (that's how the comparison is supposed
to be made case insensitive)...

The problem is that the some the regex and strings functions used are
not UTF-8 aware/friendly.

The PHP function preg_split is used when mb_split should be used,
strlower (used to make the category comparison with the translation) is
used when mb_strlower should be used.

(The "mb" prefix is for multibyte which UTF-8 use to represent a
character as soon as it is not US-ASCII...)

There might be others, I'm still trying to find more info on the other
regex functions used to confirm whether they are UTF-8 aware/friendly or
if they need additional parameters or initialization (which even
mb_strlower does) to make them work...

I'll get back to you as soon as I have more info or something to test...

Thank you and have a nice day!

Nicolas

_______________________________________________
Mythtv-translators mailing list
Mythtv-translators@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-translators