Mailing List Archive

Getting the current Freeview icons
The FreeviewHD EPG downloadable from here:

https://nzxmltv.github.io/xmltv/guide.xml

is created by scraping the freeviewnz.tv site's guide pages here:

https://freeviewnz.tv/tvguide/whats-on/

and I noticed that it also scrapes the icon URLs for the channels.
Since my channel icons are well out of date, I wanted to update them
to the same ones used by freeviewnz.tv. So I wrote some Python to
parse the guide.xml file and extract the data I wanted:

http://www.jsw.gen.nz/mythtv/get-freeview-icons.py

To use this, do this:

cd /tmp
wget http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
chmod +x get-freeview-icons.py
wget https://nzxmltv.github.io/xmltv/guide.xml
./get-freeview-icons.py guide.xml
chmod +x download-icons.sh

This will read the guide.xml file and create these two files from it:

download-icons.sh
freeview-icons.sql

The download-icons.sh file is a script containing wget commands to
download the icons. To use it, do this:

cd /home/mythtv/.mythtv/channels
/tmp/download-icons.sh
chown mythtv:mythtv *
chmod u=rw,g=rw,o=r *

Then you need to update your channel table to use the new icons. You
can do this manually, but it would be very tedious. So that is what
the freeview-icons.sql file is for - it contains the SQL to create a
new freeview-icons table in your database that contains the data
needed to use an update command to change all the icons automatically.

To use it, first backup your channel table in case there are any
problems:

sudo su
cd /tmp
mysqldump mythconverg channel >channel.sql
exit

Should you need to restore the old channel table, you need to first
shut down mythbackend and then do these commands:

sudo su
cd /tmp
mysql mythconverg
drop table channel;
source channel.sql
quit
exit

Now create the freeview_icons table:

sudo su
mysql mythconverg <freeview-icons.sql

Then you can look at the freeview_icons table to see if you think it
is ok:

select * from freeview_icons;

If that is good, you can then do the icon updates. First display the
current icons:

select channum,callsign,icon from channel where sourceid=1 order by
channum+0;

You will need to change the "sourceid=1" in this and subsequent SQL
commands to the correct sourceid number for your database.

Now do the update:

update channel c set icon=(select icon from freeview_icons f where
c.channum=f.channum) where sourceid=1 and (select count(*) from
freeview_icons f where c.channum=f.channum)!=0;

Check the icons by displaying them again:

select channum,callsign,icon from channel where sourceid=1 order by
channum+0;

Then tidy up and exit:

drop table freeview_icons;
quit
exit

Now check the icons that show in mythfrontend in the Guide. If you
are unhappy with the result restore your channel table as above.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: Getting the current Freeview icons [ In reply to ]
On Sat, 16 Apr 2022 at 00:42, Stephen Worthington
<stephen_agent@jsw.gen.nz> wrote:
>
> The FreeviewHD EPG downloadable from here:
>
> https://nzxmltv.github.io/xmltv/guide.xml
>
> is created by scraping the freeviewnz.tv site's guide pages here:
>
> https://freeviewnz.tv/tvguide/whats-on/
>
> and I noticed that it also scrapes the icon URLs for the channels.
> Since my channel icons are well out of date, I wanted to update them
> to the same ones used by freeviewnz.tv. So I wrote some Python to
> parse the guide.xml file and extract the data I wanted:
>
> http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
>
> To use this, do this:
>
> cd /tmp
> wget http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
> chmod +x get-freeview-icons.py
> wget https://nzxmltv.github.io/xmltv/guide.xml
> ./get-freeview-icons.py guide.xml
> chmod +x download-icons.sh
>
> This will read the guide.xml file and create these two files from it:
>
> download-icons.sh
> freeview-icons.sql
>
> The download-icons.sh file is a script containing wget commands to
> download the icons. To use it, do this:
>
> cd /home/mythtv/.mythtv/channels
> /tmp/download-icons.sh
> chown mythtv:mythtv *
> chmod u=rw,g=rw,o=r *
>
> Then you need to update your channel table to use the new icons. You
> can do this manually, but it would be very tedious. So that is what
> the freeview-icons.sql file is for - it contains the SQL to create a
> new freeview-icons table in your database that contains the data
> needed to use an update command to change all the icons automatically.
>
> To use it, first backup your channel table in case there are any
> problems:
>
> sudo su
> cd /tmp
> mysqldump mythconverg channel >channel.sql
> exit
>
> Should you need to restore the old channel table, you need to first
> shut down mythbackend and then do these commands:
>
> sudo su
> cd /tmp
> mysql mythconverg
> drop table channel;
> source channel.sql
> quit
> exit
>
> Now create the freeview_icons table:
>
> sudo su
> mysql mythconverg <freeview-icons.sql
>
> Then you can look at the freeview_icons table to see if you think it
> is ok:
>
> select * from freeview_icons;
>
> If that is good, you can then do the icon updates. First display the
> current icons:
>
> select channum,callsign,icon from channel where sourceid=1 order by
> channum+0;
>
> You will need to change the "sourceid=1" in this and subsequent SQL
> commands to the correct sourceid number for your database.
>
> Now do the update:
>
> update channel c set icon=(select icon from freeview_icons f where
> c.channum=f.channum) where sourceid=1 and (select count(*) from
> freeview_icons f where c.channum=f.channum)!=0;
>
> Check the icons by displaying them again:
>
> select channum,callsign,icon from channel where sourceid=1 order by
> channum+0;
>
> Then tidy up and exit:
>
> drop table freeview_icons;
> quit
> exit
>
> Now check the icons that show in mythfrontend in the Guide. If you
> are unhappy with the result restore your channel table as above.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Interestingly, the guide.xml file itself scraped all the icons for me
when I ran mythfilldatabase using it.

I had a full set of channel icons, but it seems to download its own
and install them for channels which match in your database and its
database.

I'm quite happy with it's selections. The ones I manually downloaded
from the Freeview NZ site (https://freeviewnz.tv/tvguide/) are
transparent PNG's, which look nice in some circumstances, but get lost
against a dark background in my MythTV setup.

On a final note about the guide data, apart from having to chance a
few channel names (M?ori TV with a macron ; ChineseTV8 ; Parliament TV
and a couple of others) it's good data.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: Getting the current Freeview icons [ In reply to ]
On Sat, 16 Apr 2022 19:13:08 +1200, you wrote:

>Interestingly, the guide.xml file itself scraped all the icons for me
>when I ran mythfilldatabase using it.
>
>I had a full set of channel icons, but it seems to download its own
>and install them for channels which match in your database and its
>database.

Yes, tv_grab_nz-py will update the channels and the icons unless you
run mythfilldatabase with the "--only-update-guide" option. So if the
channel data has the icon URLs, as the guide.xml file does, that will
be updated. But letting the EPG data update the channels has some
hazards, so I normally do not let that happen and use channel scanning
for channel updates.

>I'm quite happy with it's selections. The ones I manually downloaded
>from the Freeview NZ site (https://freeviewnz.tv/tvguide/) are
>transparent PNG's, which look nice in some circumstances, but get lost
>against a dark background in my MythTV setup.

It seems to be scraping the icons used on the freeviewnz.tv guide
pages on the left hand side for each channel.

>On a final note about the guide data, apart from having to chance a
>few channel names (M?ori TV with a macron ; ChineseTV8 ; Parliament TV
>and a couple of others) it's good data.

Now that MythTV's channel scanning is working reasonably well, it pays
to just use the name and callsign values that the scan takes from the
DVB-T streams. It looks like the NZXMLTV EPG data is not doing that,
so some of the names are a little different. And, of course, there is
still the problem that MythTV does not use UTF-8 everywhere and
characters do not display properly at times. But I believe the data
in the database is all UTF-8, so even if it may not display properly,
internally all the characters work (for searching, for example). If
you have a look at /usr/share/mythtv/sql/mc.sql, which contains the
SQL code used to create the mythconverg database, it has this line:

ALTER DATABASE mythconverg DEFAULT CHARACTER SET utf8 COLLATE
utf8_general_ci;

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: Getting the current Freeview icons [ In reply to ]
Maybe I got lucky but all I did to get the new icons (they were not all
available immediately after the recent channel changes) was, after a
rescan I ran (on the backend):-

Channel Editor > Icon Download

Easy.

On 16/04/22 00:41, Stephen Worthington wrote:
> The FreeviewHD EPG downloadable from here:
>
> https://nzxmltv.github.io/xmltv/guide.xml
>
> is created by scraping the freeviewnz.tv site's guide pages here:
>
> https://freeviewnz.tv/tvguide/whats-on/
>
> and I noticed that it also scrapes the icon URLs for the channels.
> Since my channel icons are well out of date, I wanted to update them
> to the same ones used by freeviewnz.tv. So I wrote some Python to
> parse the guide.xml file and extract the data I wanted:
>
> http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
>
> To use this, do this:
>
> cd /tmp
> wget http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
> chmod +x get-freeview-icons.py
> wget https://nzxmltv.github.io/xmltv/guide.xml
> ./get-freeview-icons.py guide.xml
> chmod +x download-icons.sh
>
> This will read the guide.xml file and create these two files from it:
>
> download-icons.sh
> freeview-icons.sql
>
> The download-icons.sh file is a script containing wget commands to
> download the icons. To use it, do this:
>
> cd /home/mythtv/.mythtv/channels
> /tmp/download-icons.sh
> chown mythtv:mythtv *
> chmod u=rw,g=rw,o=r *
>
> Then you need to update your channel table to use the new icons. You
> can do this manually, but it would be very tedious. So that is what
> the freeview-icons.sql file is for - it contains the SQL to create a
> new freeview-icons table in your database that contains the data
> needed to use an update command to change all the icons automatically.
>
> To use it, first backup your channel table in case there are any
> problems:
>
> sudo su
> cd /tmp
> mysqldump mythconverg channel >channel.sql
> exit
>
> Should you need to restore the old channel table, you need to first
> shut down mythbackend and then do these commands:
>
> sudo su
> cd /tmp
> mysql mythconverg
> drop table channel;
> source channel.sql
> quit
> exit
>
> Now create the freeview_icons table:
>
> sudo su
> mysql mythconverg <freeview-icons.sql
>
> Then you can look at the freeview_icons table to see if you think it
> is ok:
>
> select * from freeview_icons;
>
> If that is good, you can then do the icon updates. First display the
> current icons:
>
> select channum,callsign,icon from channel where sourceid=1 order by
> channum+0;
>
> You will need to change the "sourceid=1" in this and subsequent SQL
> commands to the correct sourceid number for your database.
>
> Now do the update:
>
> update channel c set icon=(select icon from freeview_icons f where
> c.channum=f.channum) where sourceid=1 and (select count(*) from
> freeview_icons f where c.channum=f.channum)!=0;
>
> Check the icons by displaying them again:
>
> select channum,callsign,icon from channel where sourceid=1 order by
> channum+0;
>
> Then tidy up and exit:
>
> drop table freeview_icons;
> quit
> exit
>
> Now check the icons that show in mythfrontend in the Guide. If you
> are unhappy with the result restore your channel table as above.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: Getting the current Freeview icons [ In reply to ]
It's really good to have the local option to get the EIT data, thanks
Stephen.

As pointed out, the external sources can't be relied upon forever, and do
occasionally go down - although it's been pretty reliable since I set it up
a few months back.



On Wed, Jul 6, 2022, 11:44 Robert Fisher <robert@fisher.net.nz> wrote:

> Maybe I got lucky but all I did to get the new icons (they were not all
> available immediately after the recent channel changes) was, after a
> rescan I ran (on the backend):-
>
> Channel Editor > Icon Download
>
> Easy.
>
> On 16/04/22 00:41, Stephen Worthington wrote:
> > The FreeviewHD EPG downloadable from here:
> >
> > https://nzxmltv.github.io/xmltv/guide.xml
> >
> > is created by scraping the freeviewnz.tv site's guide pages here:
> >
> > https://freeviewnz.tv/tvguide/whats-on/
> >
> > and I noticed that it also scrapes the icon URLs for the channels.
> > Since my channel icons are well out of date, I wanted to update them
> > to the same ones used by freeviewnz.tv. So I wrote some Python to
> > parse the guide.xml file and extract the data I wanted:
> >
> > http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
> >
> > To use this, do this:
> >
> > cd /tmp
> > wget http://www.jsw.gen.nz/mythtv/get-freeview-icons.py
> > chmod +x get-freeview-icons.py
> > wget https://nzxmltv.github.io/xmltv/guide.xml
> > ./get-freeview-icons.py guide.xml
> > chmod +x download-icons.sh
> >
> > This will read the guide.xml file and create these two files from it:
> >
> > download-icons.sh
> > freeview-icons.sql
> >
> > The download-icons.sh file is a script containing wget commands to
> > download the icons. To use it, do this:
> >
> > cd /home/mythtv/.mythtv/channels
> > /tmp/download-icons.sh
> > chown mythtv:mythtv *
> > chmod u=rw,g=rw,o=r *
> >
> > Then you need to update your channel table to use the new icons. You
> > can do this manually, but it would be very tedious. So that is what
> > the freeview-icons.sql file is for - it contains the SQL to create a
> > new freeview-icons table in your database that contains the data
> > needed to use an update command to change all the icons automatically.
> >
> > To use it, first backup your channel table in case there are any
> > problems:
> >
> > sudo su
> > cd /tmp
> > mysqldump mythconverg channel >channel.sql
> > exit
> >
> > Should you need to restore the old channel table, you need to first
> > shut down mythbackend and then do these commands:
> >
> > sudo su
> > cd /tmp
> > mysql mythconverg
> > drop table channel;
> > source channel.sql
> > quit
> > exit
> >
> > Now create the freeview_icons table:
> >
> > sudo su
> > mysql mythconverg <freeview-icons.sql
> >
> > Then you can look at the freeview_icons table to see if you think it
> > is ok:
> >
> > select * from freeview_icons;
> >
> > If that is good, you can then do the icon updates. First display the
> > current icons:
> >
> > select channum,callsign,icon from channel where sourceid=1 order by
> > channum+0;
> >
> > You will need to change the "sourceid=1" in this and subsequent SQL
> > commands to the correct sourceid number for your database.
> >
> > Now do the update:
> >
> > update channel c set icon=(select icon from freeview_icons f where
> > c.channum=f.channum) where sourceid=1 and (select count(*) from
> > freeview_icons f where c.channum=f.channum)!=0;
> >
> > Check the icons by displaying them again:
> >
> > select channum,callsign,icon from channel where sourceid=1 order by
> > channum+0;
> >
> > Then tidy up and exit:
> >
> > drop table freeview_icons;
> > quit
> > exit
> >
> > Now check the icons that show in mythfrontend in the Guide. If you
> > are unhappy with the result restore your channel table as above.
> >
> > _______________________________________________
> > mythtvnz mailing list
> > mythtvnz@lists.ourshack.com
> > https://lists.ourshack.com/mailman/listinfo/mythtvnz
> > Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>