Mailing List Archive

Hi,

Am Sonntag, 17. November 2002 14:44 schrieb David Klotz:
> hi everyone,
> i'm experiencing exactly the same problems as
> Daniel Paessler reported some time ago on this list (i found that post
> in the archives):
>
> i'm in germany and i have got cable, and when i run mythfilldatabase
[snip]
> my guess (because both affected users are using tv_grab_de) is,
> that there could be a problem importing the xmltv file into the database
> because of the german umlauts (ä,ö,ü,ß) in it or something like that.

No, its because tv_grab_de is returning faulty data to mythfilldb.

I have made a patch for tv_grab_de to fix this, but I will change to a new
system, using nxtvepg for retrieving the tv.xml files.

This is the patch, I hope it works.

--- tv_grab_de 2002-09-20 00:12:36.000000000 +0200
+++ /home/chris/misc/make/MYTHTV/xmltv-0.5.2/grab/de/tv_grab_de 2002-11-17
01:16:32.000000000 +0100
@@ -150,6 +150,8 @@
my $bar = new Term::ProgressBar('downloading listings', scalar @to_get)
if $have_bar;
my @listingses;
+
+
foreach (@to_get) {
my $xml;
{
@@ -160,12 +162,41 @@

# Fix empty 'length' elements that wrongly appear in the XML.
$xml =~ s!<length[^>]*></length>!!g;
+ $xml =~ s/\ \+0100/00 +0100/g;
+
+ $xml =~ s/RTL\ 2/RTL_2/g;
+ $xml =~ s/Kabel\ 1/Kabel_1/g;
+ $xml =~ s/Super\ RTL/Super_RTL/g;
+ $xml =~ s/NORD\ 3/NORD_3/g;
+ $xml =~ s/Arte\ \(ge\)/Arte/g;
+ $xml =~ s/Berlin\ 1/Berlin_1/g;
+ $xml =~ s/Hessen\ Fernse/Hessen_Fernse/g;
+ $xml =~ s/ORF\ 1/ORF_1/g;
+ $xml =~ s/ORF\ 2/ORF_2/g;
+ $xml =~ s/SF\ 1/SF_1/g;
+ $xml =~ s/SF\ 2/SF_2/g;
+ $xml =~ s/Tele\ Club/Tele_Club/g;
+ $xml =~ s/Bayern3/Bayern_3/g;
+
t 'after removing empty <length>: ' . d $xml;
}

push @listingses, XMLTV::parse($xml);
update $bar if $have_bar;
}
+
+my $file = '/tmp/channels'; # Name the file
+open(CHANFILE, $file); # Open the file
+my $line;
+foreach my $l (<CHANFILE>) {
+ $line .= $l;
+}
+
+close(CHANFILE);
+
+push @listingses, XMLTV::parse($line);
+
+
my %w_args = ();
if (defined $opt_output) {
my $fh = new IO::File ">$opt_output";

take care, have fun
/christian

On Son, 2002-11-17 at 15:02, Christian Hoenig wrote:
> No, its because tv_grab_de is returning faulty data to mythfilldb.
> I have made a patch for tv_grab_de to fix this, but I will change to a new
> system, using nxtvepg for retrieving the tv.xml files.

hmm, on the freevo site it says (or said, i can't find it in their new
documentation wikiwiki) that for german users you don't need xmltv, you
could just grab the tv listings from the web like that:
wget http://yasd.cc/xmltv/tv_`date +%Y%m%d`.xml.gz --output-document=- |
gzip -d > /tmp/TV.xml

is there a way to integrate that method into MythTV?

> This is the patch, I hope it works.

i have problems applying your patch, if i apply it using "patch -u
tv_grab_de tvgrabde.patch" it says:
patching file tv_grab_de
Hunk #1 succeeded at 153 (offset 3 lines).
Hunk #2 FAILED at 165.
1 out of 2 hunks FAILED -- saving rejects to file tv_grab_de.rej

and if i use "patch -l -u tv_grab_de tvgrabde.patch", the patch applies
fine, but after it, tv_grab_de seems to be broken, it gives the
following output:
downloading listings: ##################################################
readline() on closed filehandle CHANFILE at ./tv_grab_de line 194.
Use of uninitialized value in pattern match (m//) at
/usr/share/perl5/XMLTV.pm line 218.
no <tv> element found at ./tv_grab_de line 200

am i doing something wrong when applying your patch, or is there a
problem in the patch?

cya,
david

On Sunday 17 November 2002 10:10 am, David Klotz wrote:
> On Son, 2002-11-17 at 15:02, Christian Hoenig wrote:
> > No, its because tv_grab_de is returning faulty data to mythfilldb.
> > I have made a patch for tv_grab_de to fix this, but I will change to a
> > new system, using nxtvepg for retrieving the tv.xml files.
>
> hmm, on the freevo site it says (or said, i can't find it in their new
> documentation wikiwiki) that for german users you don't need xmltv, you
> could just grab the tv listings from the web like that:
> wget http://yasd.cc/xmltv/tv_`date +%Y%m%d`.xml.gz --output-document=- |
> gzip -d > /tmp/TV.xml
>
> is there a way to integrate that method into MythTV?

All tv_grab_de does is download those exact same xml files.

> am i doing something wrong when applying your patch, or is there a
> problem in the patch?

His patch reads in another file and prepends it to the xml downloaded by
tv_grab_de.. He didn't include the other file it reads, is all =)

The main issues with the tv_grab_de data are:

- tv_grab_de's data contains no <channel> entries in the xml.
- tv_grab_de's data's timestamp format is different than all the other xmltv
grabbers. It uses YYYYMMDDhhmm instead of YYYYMMDDhhmmss.
Mysql interprets the 12 digit timestamp as YYMMDDhhmmss, messing stuff up.

Both problems _could_ be worked around in mythfilldatabase, though his patch
pretty much fixes things in the xmltv grabber, if you have the external file
that contains all the <channel> entries and stuff.

Isaac

On Son, 2002-11-17 at 19:14, Isaac Richards wrote:
> > is there a way to integrate that method into MythTV?
> All tv_grab_de does is download those exact same xml files.

ok, i didn't know that.

> Both problems _could_ be worked around in mythfilldatabase, though his patch
> pretty much fixes things in the xmltv grabber, if you have the external file
> that contains all the <channel> entries and stuff.

now i have his external channels file, which fixes tv_grab_de crashing, but
mythfillldatabase still segfaults when i run it. :(

> Isaac

cya,
david

Am Son, 2002-11-17 um 20.19 schrieb David Klotz:
> On Son, 2002-11-17 at 19:14, Isaac Richards wrote:
> > > is there a way to integrate that method into MythTV?
> > All tv_grab_de does is download those exact same xml files.
>
> ok, i didn't know that.
>
> > Both problems _could_ be worked around in mythfilldatabase, though his patch
> > pretty much fixes things in the xmltv grabber, if you have the external file
> > that contains all the <channel> entries and stuff.
>
> now i have his external channels file, which fixes tv_grab_de crashing, but
> mythfillldatabase still segfaults when i run it. :(
>
> > Isaac
>
> cya,
> david
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@snowman.net
> http://www.snowman.net/mailman/listinfo/mythtv-dev

a question:

which external channel file do you mean? where can i get it?
at the moment i see no segfault, but my output looks like it has
something to do with that channel file, here it comes:


--snip--
[paessler@iphigenie pts/4] mythfilldatabase --manual
###
### Running in manual channel configuration mode.
### This will ask you questions about every channel.
###
----------------- Start of XMLTV output -----------------
couldn't get all of 7 days, only 6
------------------ End of XMLTV output ------------------
Unknown xmltv channel identifier: 3SAT
Skipping channel.
Unknown xmltv channel identifier: ARD
Skipping channel.
.
.
.
Unknown xmltv channel identifier: ZDF
Skipping channel.
--snip--


...snipped in the middle part. it says this about every channel, so my
database is empty after that action...


daniel
--
email: daniel@paessler.org / phone: +49 3722 83884
fax: +49 89 244365117 / web: www.paessler.org

An unknown Japanese man once said ...
Ama soyia aim duhanikam ianaka tsa akaneng...!
And that is why he is still unknown.

> a question:
>
> which external channel file do you mean? where can i get it?
> at the moment i see no segfault, but my output looks like it has
> something to do with that channel file, here it comes:

this one ;-)

take care, have fun
/christian

Am Mon, 2002-11-18 um 18.33 schrieb Christian Hoenig:
> > a question:
> >
> > which external channel file do you mean? where can i get it?
> > at the moment i see no segfault, but my output looks like it has
> > something to do with that channel file, here it comes:
>
> this one ;-)
>
> take care, have fun
> /christian

ok, have it...but where to put it?

daniel

--
email: daniel@paessler.org / phone: +49 3722 83884
fax: +49 89 244365117 / web: www.paessler.org

Sie muessen einen PC nicht einschalten, um herauszufinden,
ob WINDOWS installiert ist.
Sehen Sie einfach nach, ob die Aufschrift auf der Reset-Taste
noch lesbar ist.

Hi,

> ok, have it...but where to put it?

You have to put it to /tmp.

This file is not fully accomplished, so, for example, the icons are not set
right, but you can do that on your own, its quite simple.
(you can use this http://www.tvtv.de/epg_fp/sender_images/ard.gif)

> daniel

take care, have fun
/christian

hi,

i put it in /tmp bit still get the same error about
unknown channel identifier....
what else do i have to do? i use the yesterdays cvs
without any modifications...

daniel

On Mon, 2002-11-18 at 20:36, Christian Hoenig wrote:
> Hi,
>
> > ok, have it...but where to put it?
>
> You have to put it to /tmp.
>
> This file is not fully accomplished, so, for example, the icons are not set
> right, but you can do that on your own, its quite simple.
> (you can use this http://www.tvtv.de/epg_fp/sender_images/ard.gif)
>
> > daniel
>
> take care, have fun
> /christian
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@snowman.net
> http://www.snowman.net/mailman/listinfo/mythtv-dev
--
email: daniel@paessler.org / phone: +49 3722 83884
fax: +49 89 244365117 / web: www.paessler.org

An unknown Japanese man once said ...
Ama soyia aim duhanikam ianaka tsa akaneng...!
And that is why he is still unknown.