Mailing List Archive

On 7 July 2014 11:14, Aaron Pelly <apelly@monkeymasters.co.nz> wrote:
> I'm looking for the latest versions of the NZ guide data tools. (Assuming
> they're still open and available.)
>
> Hads' repo on github was updated 2 years ago.
> http://nice.net.nz/xmltv-proc-nz seems to be even older. I've seen
> discussion of tweaks on the list since then, so I assume there are more
> current versions kicking around somewhere.
>
> I'd like to better understand the guide data I'm getting from epg.org.nz so
> I have a better understanding of why some of my recording rules work the way
> they do.
>
> There are also a couple of things I'd like to tweak.
>
> Thanks,
> Aaron.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Good luck! If you can work out why the 'Te Reo' channel isn't
inserting EPG data, I'd be most grateful! :)

I can email you my tv_grab_nz-py script if it's helpful. It works
fine, adjusted to run on a python3 and python2 setup.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 07/07/14 11:14, Aaron Pelly wrote:
> I'm looking for the latest versions of the NZ guide data tools.
> (Assuming they're still open and available.)
>
> Hads' repo on github was updated 2 years ago.
> http://nice.net.nz/xmltv-proc-nz seems to be even older. I've seen
> discussion of tweaks on the list since then, so I assume there are more
> current versions kicking around somewhere.

They really haven't been updated in that long which is pretty slack but
unfortunately I've got to do paid work :)

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 07/07/14 11:14, Aaron Pelly wrote:
> I'm looking for the latest versions of the NZ guide data tools.
> (Assuming they're still open and available.)

I should also mention, I'm currently mostly using mhegepgsnoop as I
don't have a dish.

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Mon, Jul 7, 2014 at 6:40 PM, Hadley Rich <hads@nice.net.nz> wrote:
> On 07/07/14 11:14, Aaron Pelly wrote:
>>
>> I'm looking for the latest versions of the NZ guide data tools.
>> (Assuming they're still open and available.)
>>
>> Hads' repo on github was updated 2 years ago.
>> http://nice.net.nz/xmltv-proc-nz seems to be even older. I've seen
>> discussion of tweaks on the list since then, so I assume there are more
>> current versions kicking around somewhere.
>
>
> They really haven't been updated in that long which is pretty slack but
> unfortunately I've got to do paid work :)

How about popping it on github and letting others have a play?

>
> hads
> --
> http://nice.net.nz
>
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

>> I'm looking for the latest versions of the NZ guide data tools.
>> (Assuming they're still open and available.)
>
>I should also mention, I'm currently mostly using mhegepgsnoop as I don't
have a dish.
>
>Hads
>
I'd be really interesting in hearing how you got on and what you did given
you are probably using a HDHomeRun and are in NZ, which is my setup?

Be nice not to be reliant on an external service - not that I don't
appreciate what epg.org.nz do, I think it's great.



_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 07/07/14 18:47, Nick Rout wrote:
> How about popping it on github and letting others have a play?

They have been up there for a couple years.

--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 07/07/14 11:14, Aaron Pelly wrote:
> There are also a couple of things I'd like to tweak.
TL;DR If you want/know/care you can give advice about NZ EPG data
enhancement.

So, I've made significant progress on this front, but I'm no Python
programmer. I'd like some advice from the list.

The short story is that I've updated xmltv-proc-nz to use tmdb api v3
and to use tvdb. I also cloned tv_grab_nz-py and changed it to use
mhegepgsnoop and xmltv-proc-nz. It's a bit band-aids and string at the
moment, but over the next couple of weeks I'll make it look acceptable
and work nicely. It will appear on github. (Assuming the BSD licence
allows credited, derivative works; I am presume it does but haven't
checked yet.)

Data storage advice:
I do not know why Hads wrote xmltv-proc-nz (but I wouldn't have got this
far if he hadn't!) but it has some config that it attempts to get from
the web. This makes sense, but I have no interest in maintaning a web
service for it. The things that you might want to change, like
categories, show names etc. I have currently coded in the script. This
is plain wrong. I have three ideas about where to put these, but I'm not
sure what's best.

MySQL: Easy to implement, but less easy to edit. Requires multiple
tables. Not sure if user tables are allowed in mythconverg
.mythtv/config.xml: Easy to implement and fairly easy to edit, not sure
if user data is allowed. User hints can be stored in the XML
.xmltv/some_other_file: Sounds cleanest, but it's another file to setup
during installation, which is already a painful time.

xmltv grabber / mythfilldatabase advice:
Will mythfilldatabase run automatically during a recording? Preliminary
testing indicates that mhegepgsnoop trashes an in-progress recording,
but that could be a coincidence. More testing is required.

xmltv grabber / mhegepgsnoop advice:
There's a chicken and egg thing going on with channel configuration
here: mhegepgsnoop cunningly looks in mythconverg/channel for configured
channels, but xmltv expects you to choose your channels then myth seems
to insert them in the channel table for you, if it can. This insertion
appears to be broken for me at the moment, but that is a feature to
address another day anyway. The question really is, is it acceptable for
a new grabber to only return data for channels it finds already in the
channel table, or should it use an external config file like
tv_grab_nz-py does?

I'll finish doing this for myself anyway, but if anyone else is
interested in the results of my small effort, now is a good time to
speak up.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Tue, 14 Oct 2014 00:31:27 +1300, you wrote:

>On 07/07/14 11:14, Aaron Pelly wrote:
>> There are also a couple of things I'd like to tweak.
>TL;DR If you want/know/care you can give advice about NZ EPG data
>enhancement.
>
>So, I've made significant progress on this front, but I'm no Python
>programmer. I'd like some advice from the list.
>
>The short story is that I've updated xmltv-proc-nz to use tmdb api v3
>and to use tvdb. I also cloned tv_grab_nz-py and changed it to use
>mhegepgsnoop and xmltv-proc-nz. It's a bit band-aids and string at the
>moment, but over the next couple of weeks I'll make it look acceptable
>and work nicely. It will appear on github. (Assuming the BSD licence
>allows credited, derivative works; I am presume it does but haven't
>checked yet.)
>
>Data storage advice:
>I do not know why Hads wrote xmltv-proc-nz (but I wouldn't have got this
>far if he hadn't!) but it has some config that it attempts to get from
>the web. This makes sense, but I have no interest in maintaning a web
>service for it. The things that you might want to change, like
>categories, show names etc. I have currently coded in the script. This
>is plain wrong. I have three ideas about where to put these, but I'm not
>sure what's best.
>
>MySQL: Easy to implement, but less easy to edit. Requires multiple
>tables. Not sure if user tables are allowed in mythconverg
>.mythtv/config.xml: Easy to implement and fairly easy to edit, not sure
>if user data is allowed. User hints can be stored in the XML
>.xmltv/some_other_file: Sounds cleanest, but it's another file to setup
>during installation, which is already a painful time.

I am not sure if user tables are "allowed" either, but they work just
fine without causing MythTV any problems. But doing it that way would
require people to have some method of doing SQL on the tables to edit
them, which is beyond the skill level that is aimed at for MythTV
users. And is discouraged by the devs. Personally, I think I would
prefer to use extra database tables, but for quite a few people, they
would be much happier with a text file they can edit.

Making changes in the existing config.xml is definitely not a good
idea. It gets updated by MythTV upgrades sometimes, and that would
probably lose any extra things stored there.

>xmltv grabber / mythfilldatabase advice:
>Will mythfilldatabase run automatically during a recording? Preliminary
>testing indicates that mhegepgsnoop trashes an in-progress recording,
>but that could be a coincidence. More testing is required.

My experience is that mhegepgsnoop is fine with a recording that is
already running. It attempts to tune the tuner, but if that does not
work because mythbackend has control of the tuner, it grabs using the
current tuning and that does not disturb an in-progress recording. But
if you happen to run mhegepgsnoop just before a recording starts, it
is possible that mhegepgsnoop may have control of the tuner and that
may prevent mythbackend from using it, which will probably mess up one
or more recordings. But I have never actually had this happen as I
normally only run mhegepgsnoop manually when I am unable to get
downloaded EPG.

mythfilldatabase runs whenever it is scheduled to - it does not try to
avoid recordings. I run mine as cron job at the same time each day
rather than have mythbackend schedule it. The heavy database access
it causes can potentially cause recordings on the same drive to have
gaps in them. And in my case, it definitely can affect my Sky
recordings, due to problems with the drivers used for that. As well,
I need to use a DVB-S tuner for epgsnoop to get my Sky EPG, so I
schedule mythfilldatabase runs for when no Sky recordings are
happening.

>xmltv grabber / mhegepgsnoop advice:
>There's a chicken and egg thing going on with channel configuration
>here: mhegepgsnoop cunningly looks in mythconverg/channel for configured
>channels, but xmltv expects you to choose your channels then myth seems
>to insert them in the channel table for you, if it can. This insertion
>appears to be broken for me at the moment, but that is a feature to
>address another day anyway. The question really is, is it acceptable for
>a new grabber to only return data for channels it finds already in the
>channel table, or should it use an external config file like
>tv_grab_nz-py does?

Automatic creation of channels has always been a fraught business -
allowing it to happen can cause serious problems unless the channel
updates come from a carefully controlled source such as the US
DataDirect. There is often not enough information available from
normal EPG sources to create a full channel setup in the database. The
channel number is not usually available, for example. I run
mythfilldatabase with the --only-update-guide option to prevent it
from doing any channel updates. I do all my channel updates manually.

>I'll finish doing this for myself anyway, but if anyone else is
>interested in the results of my small effort, now is a good time to
>speak up.
>
>Aaron.

BTW Have you converted to Python 3? Python 2 is deprecated and going
away eventually in Ubuntu.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 00:31, Aaron Pelly wrote:
> I do not know why Hads wrote xmltv-proc-nz (but I wouldn't have got this
> far if he hadn't!) but it has some config that it attempts to get from
> the web. This makes sense, but I have no interest in maintaning a web
> service for it. The things that you might want to change, like
> categories, show names etc. I have currently coded in the script. This
> is plain wrong. I have three ideas about where to put these, but I'm not
> sure what's best.

The idea was that everyone could contribute a little bit to the data and
then everyone would have better data. I don't think I ever made the
contribution easy enough though, as apart from one or two people it
never happened.

You also have to keep in mind that I didn't design it just for MythTV,
it should be able to run on any system with Python so no dependancy on
MySQL etc.

I've still got an interest in EPG but an overall lack of time means I
haven't touched a thing in ages. One day.

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 00:31, Aaron Pelly wrote:
> xmltv grabber / mythfilldatabase advice:
> Will mythfilldatabase run automatically during a recording? Preliminary
> testing indicates that mhegepgsnoop trashes an in-progress recording,
> but that could be a coincidence. More testing is required.

Your message is really great news.

I have been sporadically kicking this issue around over the last couple
of years and have spent a bit of time working out how to ascertain and
predict physical tuner status so that an mheg grab will not impact
in-flight recording.

Unlike most other people I still use rb-download to grab the mmheg data
but I believe that some of the lessons I have learned should be usable
with other approaches.

I have used the mythtv services api to interrogate encoders, tuners and
upgoming recordings to establish the next 200 sec window where the 2
highest recpriority tuners are either busy or free.

I chose 200 secs because experimentation and experience has shown that
it only takes approx 50 secs to grab the whole mheg data carosel.

I prefer a busy tuner because than there is no overhead of tuning the
device before starting to grab the data.

Then I kick off the rb-download task but also spawn another thread to
monitor the downloaded data structures and flag the control task when
all directories and files have been populated.

Rb-download is a never ending program so has to be killed when you have
enough data.

I have had various versions of this running for a couple of years now
via cron at 3:00am with NO intervention and after working out a couple
of subtle bugs have had no problem with it at all. Never had a trashed
recording.

Here is the debug log data for a single run on 14 Sep this year. I
have chosen this date as English premier league was being recorded at
3:00 am.

# ##################################################################
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 CARDID: 1
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 CARDID: 2
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 CARDID: 3
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 CARDID: 4
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 CARDID: 5
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 1 CARDID: 6
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 1 CARDID: 7
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 1 CARDID: 8
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 1 CARDID: 9
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 2 CARDID: 10
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 2 CARDID: 11
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 2 CARDID: 12
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 2 CARDID: 13
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 0 RECPRIORITY: 1
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 1 RECPRIORITY: 2
RUNTIME: 03:00:03 on 14/09/2014 VIDEODEVICE: 2 RECPRIORITY: 0
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1016 SERVICEID: 1502
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1036 SERVICEID: 1501
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1003 SERVICEID: 1300
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1004 SERVICEID: 1301
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1008 SERVICEID: 1302
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1009 SERVICEID: 1303
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1018 SERVICEID: 1304
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1001 SERVICEID: 1200
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1002 SERVICEID: 1201
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1006 SERVICEID: 1206
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1007 SERVICEID: 1207
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1005 SERVICEID: 1400
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1022 SERVICEID: 1401
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1028 SERVICEID: 1403
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1010 SERVICEID: 1404
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1033 SERVICEID: 1405
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1012 SERVICEID: 1407
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1030 SERVICEID: 1408
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1024 SERVICEID: 1409
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1031 SERVICEID: 1410
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1032 SERVICEID: 1411
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1035 SERVICEID: 1412
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1026 SERVICEID: 1414
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1014 SERVICEID: 1415
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1040 SERVICEID: 1416
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1039 SERVICEID: 1417
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1023 SERVICEID: 1418
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1034 SERVICEID: 1421
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1021 SERVICEID: 1503
RUNTIME: 03:00:03 on 14/09/2014 CHANID: 1020 SERVICEID: 1504
RUNTIME: 03:00:03 on 14/09/2014 SLEEPTIME 3772
ADAPTER0: Free: 0, 52192, <<<<< Free for 52192 secs
ADAPTER1: Busy: 1, 3772, 1200, <<<<< Busy for 3772 secs
ADAPTER2: Free: 0, 59212

RUNTIME: 03:00:03 on 14/09/2014 SERVICEID FOR CAPTURE: 1200
RUNTIME: 03:00:03 on 14/09/2014 COMMAND: /usr/local/bin/rb-download -a 1
-b /tmp/mheg_snoop_epg_data -f /home/tartarus/data/channels.conf 1200 >
/dev/null 2>&1
RUNTIME: 03:00:03 on 14/09/2014 DURATION:51 secs. ADAPTER:1 SERVICEID:
1200 DATATILL: 20140921


# ##################################################################

I could cut the cord to Hads EPG but have not mainly because I do not
want to lose the benefits of Hads optimisations.

So I have just been keeping my code in the bottom drawer against a
possible future copyright bloodhounds interference again.

If you want to explore the use of any of this let me know.

Regards

Tony


Afterthought: Hads original idea to Crowd-Source the coding of
categories etc was inventive and I participated for a while but soon
found that different people categorise things differently (eg One
persons drama is another persons soap) and people were stomping over
each other so it became impossible to set up the schedule your way and
keep it that way.



_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 00:31, Aaron Pelly wrote:
> On 07/07/14 11:14, Aaron Pelly wrote:
>> There are also a couple of things I'd like to tweak.
> TL;DR If you want/know/care you can give advice about NZ EPG data
> enhancement.
>
> ....
> xmltv grabber / mythfilldatabase advice:
> Will mythfilldatabase run automatically during a recording?
> Preliminary testing indicates that mhegepgsnoop trashes an in-progress
> recording, but that could be a coincidence. More testing is required.
>
I run a script every hour which runs mhegepgsnoop then mythfilldatabase
(if mhegepegsnoop finds any EPG updates). I don't think I've ever seen
any recording glitches due to this and at least once every day
mhegepgsnoop is reading from the same tuner that is recording a show.
Maybe something to do with hardware or virtual tuners or myth config is
causing your problem? I have an HVR2200.
> xmltv grabber / mhegepgsnoop advice:
> There's a chicken and egg thing going on with channel configuration
> here: mhegepgsnoop cunningly looks in mythconverg/channel for
> configured channels, but xmltv expects you to choose your channels
> then myth seems to insert them in the channel table for you, if it can.
Not sure what you mean here. I don't think mythfilldatabase adds
channels unless maybe you can tell it to? Is there a mythfilldatabase
option for that? My experience is that I need to manually add new
channels. I usually scan in mythtv and then carefully add any new
channels using the manual options in scan so I don't trash my existing
channel configs.
> This insertion appears to be broken for me at the moment, but that is
> a feature to address another day anyway. The question really is, is it
> acceptable for a new grabber to only return data for channels it finds
> already in the channel table, or should it use an external config file
> like tv_grab_nz-py does?
>
> I'll finish doing this for myself anyway, but if anyone else is
> interested in the results of my small effort, now is a good time to
> speak up.
>
> Aaron.
>


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 05:16, Stephen Worthington wrote:
> I am not sure if user tables are "allowed" either, but they work just
> fine without causing MythTV any problems. But doing it that way would
> require people to have some method of doing SQL on the tables to edit
> them, which is beyond the skill level that is aimed at for MythTV
> users.
My thoughts too.
> Making changes in the existing config.xml is definitely not a good
> idea. It gets updated by MythTV upgrades sometimes, and that would
> probably lose any extra things stored there.
>
Understood.
>> xmltv grabber / mythfilldatabase advice:
>> Will mythfilldatabase run automatically during a recording? Preliminary
>> testing indicates that mhegepgsnoop trashes an in-progress recording,
>> but that could be a coincidence. More testing is required.
> My experience is that mhegepgsnoop is fine with a recording that is
> already running.
Thanks for the data point.
> I normally only run mhegepgsnoop manually when I am unable to get
> downloaded EPG.
I figured that not relying on an external source might be helpful. For
instance, someone mentioned the Te Reo guide some time ago.
> mythfilldatabase runs whenever it is scheduled to - it does not try to
> avoid recordings. I run mine as cron job at the same time each day
> rather than have mythbackend schedule it. The heavy database access
> it causes can potentially cause recordings on the same drive to have
> gaps in them.
I understand. But, it seems bit like having a dog, and barking too.
> As well, I need to use a DVB-S tuner for epgsnoop to get my Sky EPG, so I
> schedule mythfilldatabase runs for when no Sky recordings are
> happening.
How many people are in this boat? Any thoughts?
>
> Automatic creation of channels has always been a fraught business -
> allowing it to happen can cause serious problems unless the channel
> updates come from a carefully controlled source such as the US
> DataDirect. There is often not enough information available from
> normal EPG sources to create a full channel setup in the database. The
> channel number is not usually available, for example. I run
> mythfilldatabase with the --only-update-guide option to prevent it
> from doing any channel updates. I do all my channel updates manually.
Yes. It's far from a priority for me.
> BTW Have you converted to Python 3? Python 2 is deprecated and going
> away eventually in Ubuntu.
>
No. But I will now that I know. Thanks for the heads up.

Stephen, you always take the time to make full and clear posts. Thank
you; you are very helpful.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 07:48, Hadley Rich wrote:
> On 14/10/14 00:31, Aaron Pelly wrote:
>> I do not know why Hads wrote xmltv-proc-nz (but I wouldn't have got this
>> far if he hadn't!) but it has some config that it attempts to get from
>> the web. This makes sense, but I have no interest in maintaning a web
>> service for it. The things that you might want to change, like
>> categories, show names etc. I have currently coded in the script. This
>> is plain wrong. I have three ideas about where to put these, but I'm not
>> sure what's best.
>
> The idea was that everyone could contribute a little bit to the data
> and then everyone would have better data. I don't think I ever made
> the contribution easy enough though, as apart from one or two people
> it never happened.
I guessed that. You must have put a boat load of effort into this stuff.
I don't really have the skill set to replicate what you were aiming for,
but now I'm thinking of a small text config file, so it should be quite
easy (but annoyingly manual) to share on the list.
>
> You also have to keep in mind that I didn't design it just for MythTV,
> it should be able to run on any system with Python so no dependancy on
> MySQL etc.
I'll keep that in mind.
>
> I've still got an interest in EPG but an overall lack of time means I
> haven't touched a thing in ages. One day.
>
No need for excuses! As I said; I'd still have nothing if it wasn't for
your earlier effort.


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 10:15, Tony Sauri wrote:
> On 14/10/14 00:31, Aaron Pelly wrote:
>> xmltv grabber / mythfilldatabase advice:
>> Will mythfilldatabase run automatically during a recording? Preliminary
>> testing indicates that mhegepgsnoop trashes an in-progress recording,
>> but that could be a coincidence. More testing is required.
> Your message is really great news.
>
> I have been sporadically kicking this issue around over the last couple
> of years and have spent a bit of time working out how to ascertain and
> predict physical tuner status so that an mheg grab will not impact
> in-flight recording.
Interestingly there is a thread along these lines in the international
list right now.
>
> Unlike most other people I still use rb-download to grab the mmheg data
> but I believe that some of the lessons I have learned should be usable
> with other approaches.
I have failed to hear about rb-download until now (I think). I'll see if
it has any useful fruit in it.
> I could cut the cord to Hads EPG but have not mainly because I do not
> want to lose the benefits of Hads optimisations.
Those optimisations are key.
> So I have just been keeping my code in the bottom drawer against a
> possible future copyright bloodhounds interference again.
That worries me too.
>
> If you want to explore the use of any of this let me know.
Very much!
> Afterthought: Hads original idea to Crowd-Source the coding of
> categories etc was inventive and I participated for a while but soon
> found that different people categorise things differently (eg One
> persons drama is another persons soap) and people were stomping over
> each other so it became impossible to set up the schedule your way and
> keep it that way.
With proper title optimisation and TVDB I think this will be less of an
issue. I've seen pretty good results.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 10:39, David Moore wrote:
> I run a script every hour which runs mhegepgsnoop then
> mythfilldatabase (if mhegepegsnoop finds any EPG updates). I don't
> think I've ever seen any recording glitches due to this and at least
> once every day mhegepgsnoop is reading from the same tuner that is
> recording a show. Maybe something to do with hardware or virtual
> tuners or myth config is causing your problem? I have an HVR2200.
Another data point. Maybe I was just unlucky. I must admit, I can't see
why it would cause problems. Anyway, thanks.
>> xmltv grabber / mhegepgsnoop advice:
>> There's a chicken and egg thing going on with channel configuration
>> here: mhegepgsnoop cunningly looks in mythconverg/channel for
>> configured channels, but xmltv expects you to choose your channels
>> then myth seems to insert them in the channel table for you, if it can.
> Not sure what you mean here. I don't think mythfilldatabase adds
> channels unless maybe you can tell it to? Is there a mythfilldatabase
> option for that? My experience is that I need to manually add new
> channels.
I discovered myth attempting to do it all on its own the other day. I
had some misconfigured tuners; I was trying to disable DVB-S
temporarily to diagnose reception issues (STILL unresolved, damn it!).
Myth had been happily and incorrectly adding channels for some time. It
took a while to notice because I wasn't using those tuners.

> I usually scan in mythtv and then carefully add any new channels using
> the manual options in scan so I don't trash my existing channel configs.
Tuning is still a black art to me. Everything I touch breaks something.



_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Tue, 14 Oct 2014 11:23:09 +1300, you wrote:

>On 14/10/14 05:16, Stephen Worthington wrote:

>> As well, I need to use a DVB-S tuner for epgsnoop to get my Sky EPG, so I
>> schedule mythfilldatabase runs for when no Sky recordings are
>> happening.
>How many people are in this boat? Any thoughts?

Here in NZ, I think only one or two other people are recording Sky
directly from DVB-S like I am. But people tend to keep quiet about it
as Sky are unlikely to be pleased. They want us all to pay extra for
one of their Sky PVR boxes, which are far less capable than MythTV.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 10:15, Tony Sauri wrote:
> Afterthought: Hads original idea to Crowd-Source the coding of
> categories etc was inventive and I participated for a while but soon
> found that different people categorise things differently (eg One
> persons drama is another persons soap) and people were stomping over
> each other so it became impossible to set up the schedule your way and
> keep it that way.

Thanks Tony, you were one I was thinking of when I said one or two
people were adding content.

Good point about the differing categorisations, often it's not clear
where something should be. I think the solution to that is multiple
levels of processing. Grab the crowd-sourced data initially and then
overwrite that with local preferences, you then get the benefit without
the drawback.

Easy to implement too if you just use a JSON feed for the webservice and
a local file in the same format for the overrides.

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 14/10/14 13:40, Hadley Rich wrote:
>
> Easy to implement too if you just use a JSON feed for the webservice
> and a local file in the same format for the overrides.
>
I was going to use XML, because, well, it's xmltv. Also users have
probably seen it for the myth config. JSON doesn't look pretty.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Tue, Oct 14, 2014 at 1:38 PM, Stephen Worthington <
stephen_agent@jsw.gen.nz> wrote:

> On Tue, 14 Oct 2014 11:23:09 +1300, you wrote:
>
> >On 14/10/14 05:16, Stephen Worthington wrote:
>
> >> As well, I need to use a DVB-S tuner for epgsnoop to get my Sky EPG,
> so I
> >> schedule mythfilldatabase runs for when no Sky recordings are
> >> happening.
> >How many people are in this boat? Any thoughts?
>
> Here in NZ, I think only one or two other people are recording Sky
> directly from DVB-S like I am. But people tend to keep quiet about it
> as Sky are unlikely to be pleased. They want us all to pay extra for
> one of their Sky PVR boxes, which are far less capable than MythTV.
>

I'm grabbing the Sky EPG data but doing the recording via an analogue
capture card (don't think I have the right Sky card to capture directly).
If Sky happens to be recording at the time the tuner will already be tuned
so the data grabber will work anyway. If it didn't work it's only going to
miss one day so it's not a problem. Even so I grab data at 4am so I'm
almost never recording then.

Cheers,
Steve

On 14/10/14 05:16, Stephen Worthington wrote:
> BTW Have you converted to Python 3? Python 2 is deprecated and going
> away eventually in Ubuntu.
This has caused some issues. (Probably) nothing serious, but I know
nothing of the Python ecosystem. I have had to port both the movie and
tv api modules that I was using.

I know this is not the list for tech-python-speak, but does anyone here
have experience publishing a Python module?

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 16/10/14 23:01, Aaron Pelly wrote:
> On 14/10/14 05:16, Stephen Worthington wrote:
>> BTW Have you converted to Python 3? Python 2 is deprecated and going
>> away eventually in Ubuntu.
> This has caused some issues. (Probably) nothing serious, but I know
> nothing of the Python ecosystem. I have had to port both the movie and
> tv api modules that I was using.
>
> I know this is not the list for tech-python-speak, but does anyone here
> have experience publishing a Python module?
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/


Did you see this thread across on the mythtv-users list?

http://www.gossamer-threads.com/lists/mythtv/users/577769

Porting to Python3 might not be important in the short term.

Tony

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 16/10/14 23:30, Tony Sauri wrote:
> On 16/10/14 23:01, Aaron Pelly wrote:
>> On 14/10/14 05:16, Stephen Worthington wrote:
>>> BTW Have you converted to Python 3? Python 2 is deprecated and going
>>> away eventually in Ubuntu.
>> This has caused some issues. (Probably) nothing serious, but I know
>> nothing of the Python ecosystem. I have had to port both the movie and
>> tv api modules that I was using.
>>
>> I know this is not the list for tech-python-speak, but does anyone here
>> have experience publishing a Python module?
>>
>>
>>
>> Did you see this thread across on the mythtv-users list?
>>
>> http://www.gossamer-threads.com/lists/mythtv/users/577769
>>
>> Porting to Python3 might not be important in the short term.
>>
>>
I did. And I'm still in two minds.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 17 Oct 2014 07:19:43 +1300, you wrote:

>On 16/10/14 23:30, Tony Sauri wrote:
>> On 16/10/14 23:01, Aaron Pelly wrote:
>>> On 14/10/14 05:16, Stephen Worthington wrote:
>>>> BTW Have you converted to Python 3? Python 2 is deprecated and going
>>>> away eventually in Ubuntu.
>>> This has caused some issues. (Probably) nothing serious, but I know
>>> nothing of the Python ecosystem. I have had to port both the movie and
>>> tv api modules that I was using.
>>>
>>> I know this is not the list for tech-python-speak, but does anyone here
>>> have experience publishing a Python module?
>>>
>>>
>>>
>>> Did you see this thread across on the mythtv-users list?
>>>
>>> http://www.gossamer-threads.com/lists/mythtv/users/577769
>>>
>>> Porting to Python3 might not be important in the short term.
>>>
>>>
>I did. And I'm still in two minds.

It looks to me like there are a chunk of Python 2 users who are going
to keep on using Python 2 until forced out of their rut. But the
Python devs seem to be very clear about what is happening - no further
Python 2 development, only bug fixes. There are already quite useful
changes in the standard Python 3 libraries that will never be in
Python 2, and the clean unicode support seems to me to be an excellent
thing, as we have been bitten by character set problems with our NZ
EPG more than once. This situation all reminds me of IPv4 vs IPv6. We
should all be converted to IPv6 by now, but endless people are just
ignoring the problem until it bites them badly enough. And with
Python, some of the players are companies big enough to be able to
afford the cost of actually forking Python 2 and keeping on developing
it, should they choose to. The basic problem seems to be that there
are many more packages available for Python 2 than Python 3. So
anyone whose favourite package is 2 only is likely to not want to
change until that package does.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 17/10/14 14:06, Stephen Worthington wrote:
> It looks to me like there are a chunk of Python 2 users who are going
> to keep on using Python 2 until forced out of their rut. But the
> Python devs seem to be very clear about what is happening - no further
> Python 2 development, only bug fixes. There are already quite useful
> changes in the standard Python 3 libraries that will never be in
> Python 2, and the clean unicode support seems to me to be an excellent
> thing, as we have been bitten by character set problems with our NZ
> EPG more than once. This situation all reminds me of IPv4 vs IPv6. We
> should all be converted to IPv6 by now, but endless people are just
> ignoring the problem until it bites them badly enough. And with
> Python, some of the players are companies big enough to be able to
> afford the cost of actually forking Python 2 and keeping on developing
> it, should they choose to.
Yes. All that!
> The basic problem seems to be that there
> are many more packages available for Python 2 than Python 3. So
> anyone whose favourite package is 2 only is likely to not want to
> change until that package does.
This too. I couldn't find a anything that looked like a decent
replacement for tvdb_api, which I already had installed, so I guess is
probably favoured by Myth.

Porting this to Python3 has slowed me down by a few days. It doesn't
help that I have no idea what I'm doing :)

I think I'm nearly there.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 07/07/14 11:14, Aaron Pelly wrote:
> There are also a couple of things I'd like to tweak.

For those that are interested, I have xmltv-proc-nz working with Python
3: https://github.com/apelly/xmltv-tools

At this stage it it's still a bit of a pain to set up. You will need to
install the tmdb & tvdb apis from my repo too. I have not ported any of
the installation or testing scripts for them. This may or may not cause
you a problem.

It's still early days, but advice and feedback would be appreciated.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

TL;DR Comments requested for a replacement xmltv grabber for NZ

I've been thinking about how to string all this together in an xmltv
friendly way.

My thoughts, based on my progress so far:
1) Change mhegepgsnoop to invent xmltv channel names based on somehting
like trim(name)+'.freeview.nz' and return all channel data, regardless
of which channels interest you. We have to decode it all anyway. This
means mhegepgsnoop no longer needs configuration. But will probably mean
renaming some of your channels in Myth. Can also make it handle utf-8
characters properly.

2) Process the data as per the xmltv spec, to exclude everything that is
not interesting. This should be easy enough to do by modifying an
existing grabber. The recommended new grabbers do seem more
feature-complete than our current one.

3) Within the modified grabber, call xmltv-proc-nz to augment any
remaining data. This processes the minimum amount of data, which is a
good thing.

I'm kind of reluctant to make yet another copy of an mheg reader,
knowing that my copy will surely fall into disrepair at some stage.
Perhaps the author would care to comment? On the other hand, the code
could really do with some comments and a bit of a clean up. Caching
would be good too, for testing. And it might be interesting to make it
Python 3 as well.

As for utf-8; I haven't had any to work with yet, because they have been
missing in the guide data. I am confident this can be made to work
properly with only minor changes to existing code.

I feel like I'm very close to something good here. xmltv-proc-nz
probably needs more options, but, from where I am, this should be
crazy-easy to implement.

Anyway, I'm kind of thinking out loud here, but if anyone has an opinion
I'm all ears.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 18/10/14 20:55, Aaron Pelly wrote:
> TL;DR Comments requested for a replacement xmltv grabber for NZ
>
> I've been thinking about how to string all this together in an xmltv
> friendly way.
>
> My thoughts, based on my progress so far:
> 1) Change mhegepgsnoop to invent xmltv channel names based on
> somehting like trim(name)+'.freeview.nz' and return all channel data,
> regardless of which channels interest you. We have to decode it all
> anyway. This means mhegepgsnoop no longer needs configuration. But
> will probably mean renaming some of your channels in Myth. Can also
> make it handle utf-8 characters properly.
>
> 2) Process the data as per the xmltv spec, to exclude everything that
> is not interesting. This should be easy enough to do by modifying an
> existing grabber. The recommended new grabbers do seem more
> feature-complete than our current one.
>
> 3) Within the modified grabber, call xmltv-proc-nz to augment any
> remaining data. This processes the minimum amount of data, which is a
> good thing.
>
> I'm kind of reluctant to make yet another copy of an mheg reader,
> knowing that my copy will surely fall into disrepair at some stage.
> Perhaps the author would care to comment? On the other hand, the code
> could really do with some comments and a bit of a clean up. Caching
> would be good too, for testing. And it might be interesting to make it
> Python 3 as well.
>
> As for utf-8; I haven't had any to work with yet, because they have
> been missing in the guide data. I am confident this can be made to
> work properly with only minor changes to existing code.
>
> I feel like I'm very close to something good here. xmltv-proc-nz
> probably needs more options, but, from where I am, this should be
> crazy-easy to implement.
>
> Anyway, I'm kind of thinking out loud here, but if anyone has an
> opinion I'm all ears.
>
Hi Aaron,

I'm the primary author of mhegepgsnoop. Others have made useful
contributions and I'm happy to accept changes but I have one
non-negotiable rule. Any changes must be backward compatible. So if you
want to _extend_ mhegepgsnoop in some way then go for it. Just add
another command line option and the help for that option. I'm happy to
help you with this.

I would strongly recommend against forking the project because I won't
support such a fork. Your comment about not making another mheg reader
is probably wiser than you think. Getting and parsing the mheg data was
by far the trickiest part of the code. So if it needs to change in
future there is quite a steep learning curve to acheive reasonable
understanding of this part of the code. But, hey, if you're up for the
challenge then go for it. :) Maybe you'll work out a better way to get
the data.

BTW if you like C (C++?) then have a look at the rb-download source to
see how it gets the mheg data.

Offer to clean up code and add more comments (where really useful)
gratefully accepted. :) Actually I know the code is a bit spaghetti.
Knowing what I know now I would write it much differently if I started
now. But it ain't broke so I don't fix it.

BTW I really don't see Python 3 compatibility as something that needs to
be tackled soon. Fine if you want to do it but it seems to me that
Python 2 will be around for a while yet. Just check out the number of
libs available for 2 vs 3. 2 won't go away until 3 has a similar number
of libs available.

Cheers
David

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 18/10/14 20:55, Aaron Pelly wrote:
> TL;DR Comments requested for a replacement xmltv grabber for NZ
>
> I've been thinking about how to string all this together in an xmltv
> friendly way.
>
> My thoughts, based on my progress so far:
> 1) Change mhegepgsnoop to invent xmltv channel names based on
> somehting like trim(name)+'.freeview.nz' and return all channel data,
> regardless of which channels interest you. We have to decode it all
> anyway. This means mhegepgsnoop no longer needs configuration. But
> will probably mean renaming some of your channels in Myth. Can also
> make it handle utf-8 characters properly.
Forgot to mention that mhegepgsnoop does support no config. It has the
option to use fuzzy string matching of channel names in the mheg epg vs
channel names in mythtv. So you just need to set up your myth channel
names similar to the mheg names. Use the verbose log output to see the
mheg names. Works for me.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 11:33, David Moore wrote:
> On 18/10/14 20:55, Aaron Pelly wrote:
>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>
>> I've been thinking about how to string all this together in an xmltv
>> friendly way.
>>
>> My thoughts, based on my progress so far:
>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>> somehting like trim(name)+'.freeview.nz' and return all channel data,
>> regardless of which channels interest you. We have to decode it all
>> anyway. This means mhegepgsnoop no longer needs configuration. But
>> will probably mean renaming some of your channels in Myth. Can also
>> make it handle utf-8 characters properly.
>>
>> 2) Process the data as per the xmltv spec, to exclude everything that
>> is not interesting. This should be easy enough to do by modifying an
>> existing grabber. The recommended new grabbers do seem more
>> feature-complete than our current one.
>>
>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>> remaining data. This processes the minimum amount of data, which is a
>> good thing.
>>
>> I'm kind of reluctant to make yet another copy of an mheg reader,
>> knowing that my copy will surely fall into disrepair at some stage.
>> Perhaps the author would care to comment? On the other hand, the code
>> could really do with some comments and a bit of a clean up. Caching
>> would be good too, for testing. And it might be interesting to make
>> it Python 3 as well.
>>
>> As for utf-8; I haven't had any to work with yet, because they have
>> been missing in the guide data. I am confident this can be made to
>> work properly with only minor changes to existing code.
>>
>> I feel like I'm very close to something good here. xmltv-proc-nz
>> probably needs more options, but, from where I am, this should be
>> crazy-easy to implement.
>>
>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>> opinion I'm all ears.
>>
> Hi Aaron,
>
> I'm the primary author of mhegepgsnoop. Others have made useful
> contributions and I'm happy to accept changes but I have one
> non-negotiable rule. Any changes must be backward compatible. So if
> you want to _extend_ mhegepgsnoop in some way then go for it. Just add
> another command line option and the help for that option.

Awesome! This is great news.

> Getting and parsing the mheg data was by far the trickiest part of the
> code. So if it needs to change in future there is quite a steep
> learning curve to acheive reasonable understanding of this part of the
> code.

Now the hard part has been done I might see what else is in there.

> But, hey, if you're up for the challenge then go for it. :) Maybe
> you'll work out a better way to get the data.

Ha!

> Offer to clean up code and add more comments (where really useful)
> gratefully accepted. :) Actually I know the code is a bit spaghetti.
> Knowing what I know now I would write it much differently if I started
> now.

My code rarely starts with a good plan, and I've been writing (not
Python!) for years. Hard problems like this are the best at causing
temporary code that "I'll get back to"

> it ain't broke so I don't fix it.

Good reasoning.

>
> BTW I really don't see Python 3 compatibility as something that needs
> to be tackled soon. Fine if you want to do it but it seems to me that
> Python 2 will be around for a while yet. Just check out the number of
> libs available for 2 vs 3. 2 won't go away until 3 has a similar
> number of libs available.

Yes, but I can't write Python, so I /really/ don't want to be able to
write Python 2 and Python 3.

One issue though: Do you see porting to Python 3 as breaking backwards
compatibility?

Anyway, as I said; it's early days. I'll have a better opinion once I've
hacked at the code a bit.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 13:08, Aaron Pelly wrote:
> On 19/10/14 11:33, David Moore wrote:
>> On 18/10/14 20:55, Aaron Pelly wrote:
>>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>>
>>> I've been thinking about how to string all this together in an xmltv
>>> friendly way.
>>>
>>> My thoughts, based on my progress so far:
>>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>>> somehting like trim(name)+'.freeview.nz' and return all channel
>>> data, regardless of which channels interest you. We have to decode
>>> it all anyway. This means mhegepgsnoop no longer needs
>>> configuration. But will probably mean renaming some of your channels
>>> in Myth. Can also make it handle utf-8 characters properly.
>>>
>>> 2) Process the data as per the xmltv spec, to exclude everything
>>> that is not interesting. This should be easy enough to do by
>>> modifying an existing grabber. The recommended new grabbers do seem
>>> more feature-complete than our current one.
>>>
>>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>>> remaining data. This processes the minimum amount of data, which is
>>> a good thing.
>>>
>>> I'm kind of reluctant to make yet another copy of an mheg reader,
>>> knowing that my copy will surely fall into disrepair at some stage.
>>> Perhaps the author would care to comment? On the other hand, the
>>> code could really do with some comments and a bit of a clean up.
>>> Caching would be good too, for testing. And it might be interesting
>>> to make it Python 3 as well.
>>>
>>> As for utf-8; I haven't had any to work with yet, because they have
>>> been missing in the guide data. I am confident this can be made to
>>> work properly with only minor changes to existing code.
>>>
>>> I feel like I'm very close to something good here. xmltv-proc-nz
>>> probably needs more options, but, from where I am, this should be
>>> crazy-easy to implement.
>>>
>>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>>> opinion I'm all ears.
>>>
>> Hi Aaron,
>>
>> I'm the primary author of mhegepgsnoop. Others have made useful
>> contributions and I'm happy to accept changes but I have one
>> non-negotiable rule. Any changes must be backward compatible. So if
>> you want to _extend_ mhegepgsnoop in some way then go for it. Just
>> add another command line option and the help for that option.
>
> Awesome! This is great news.
>
>> Getting and parsing the mheg data was by far the trickiest part of
>> the code. So if it needs to change in future there is quite a steep
>> learning curve to acheive reasonable understanding of this part of
>> the code.
>
> Now the hard part has been done I might see what else is in there.
>
>> But, hey, if you're up for the challenge then go for it. :) Maybe
>> you'll work out a better way to get the data.
>
> Ha!
>
>> Offer to clean up code and add more comments (where really useful)
>> gratefully accepted. :) Actually I know the code is a bit spaghetti.
>> Knowing what I know now I would write it much differently if I
>> started now.
>
> My code rarely starts with a good plan, and I've been writing (not
> Python!) for years. Hard problems like this are the best at causing
> temporary code that "I'll get back to"
>
>> it ain't broke so I don't fix it.
>
> Good reasoning.
>
>>
>> BTW I really don't see Python 3 compatibility as something that needs
>> to be tackled soon. Fine if you want to do it but it seems to me that
>> Python 2 will be around for a while yet. Just check out the number of
>> libs available for 2 vs 3. 2 won't go away until 3 has a similar
>> number of libs available.
>
> Yes, but I can't write Python, so I /really/ don't want to be able to
> write Python 2 and Python 3.
>
> One issue though: Do you see porting to Python 3 as breaking backwards
> compatibility?
>
> Anyway, as I said; it's early days. I'll have a better opinion once
> I've hacked at the code a bit.
>
> Aaron.
>
My understanding is that Python 3 is deliberately not backward
compatible with Python 2. So I guess it would be tedious to refactor the
code so it runs on both 2 and 3. I think I'm saying that support for 3
would be a fork of the 2 code and both would have to be maintained in
parallel for a while until everyone moves to 3.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 13:08, Aaron Pelly wrote:
> On 19/10/14 11:33, David Moore wrote:
>> On 18/10/14 20:55, Aaron Pelly wrote:
>>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>>
>>> I've been thinking about how to string all this together in an xmltv
>>> friendly way.
>>>
>>> My thoughts, based on my progress so far:
>>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>>> somehting like trim(name)+'.freeview.nz' and return all channel
>>> data, regardless of which channels interest you. We have to decode
>>> it all anyway. This means mhegepgsnoop no longer needs
>>> configuration. But will probably mean renaming some of your channels
>>> in Myth. Can also make it handle utf-8 characters properly.
>>>
>>> 2) Process the data as per the xmltv spec, to exclude everything
>>> that is not interesting. This should be easy enough to do by
>>> modifying an existing grabber. The recommended new grabbers do seem
>>> more feature-complete than our current one.
>>>
>>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>>> remaining data. This processes the minimum amount of data, which is
>>> a good thing.
>>>
>>> I'm kind of reluctant to make yet another copy of an mheg reader,
>>> knowing that my copy will surely fall into disrepair at some stage.
>>> Perhaps the author would care to comment? On the other hand, the
>>> code could really do with some comments and a bit of a clean up.
>>> Caching would be good too, for testing. And it might be interesting
>>> to make it Python 3 as well.
>>>
>>> As for utf-8; I haven't had any to work with yet, because they have
>>> been missing in the guide data. I am confident this can be made to
>>> work properly with only minor changes to existing code.
>>>
>>> I feel like I'm very close to something good here. xmltv-proc-nz
>>> probably needs more options, but, from where I am, this should be
>>> crazy-easy to implement.
>>>
>>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>>> opinion I'm all ears.
>>>
>> Hi Aaron,
>>
>> I'm the primary author of mhegepgsnoop. Others have made useful
>> contributions and I'm happy to accept changes but I have one
>> non-negotiable rule. Any changes must be backward compatible. So if
>> you want to _extend_ mhegepgsnoop in some way then go for it. Just
>> add another command line option and the help for that option.
>
> Awesome! This is great news.
>
>> Getting and parsing the mheg data was by far the trickiest part of
>> the code. So if it needs to change in future there is quite a steep
>> learning curve to acheive reasonable understanding of this part of
>> the code.
>
> Now the hard part has been done I might see what else is in there.
>
>> But, hey, if you're up for the challenge then go for it. :) Maybe
>> you'll work out a better way to get the data.
>
> Ha!
>
>> Offer to clean up code and add more comments (where really useful)
>> gratefully accepted. :) Actually I know the code is a bit spaghetti.
>> Knowing what I know now I would write it much differently if I
>> started now.
>
> My code rarely starts with a good plan, and I've been writing (not
> Python!) for years. Hard problems like this are the best at causing
> temporary code that "I'll get back to"
>
>> it ain't broke so I don't fix it.
>
> Good reasoning.
>
>>
>> BTW I really don't see Python 3 compatibility as something that needs
>> to be tackled soon. Fine if you want to do it but it seems to me that
>> Python 2 will be around for a while yet. Just check out the number of
>> libs available for 2 vs 3. 2 won't go away until 3 has a similar
>> number of libs available.
>
> Yes, but I can't write Python, so I /really/ don't want to be able to
> write Python 2 and Python 3.
>
> One issue though: Do you see porting to Python 3 as breaking backwards
> compatibility?
>
> Anyway, as I said; it's early days. I'll have a better opinion once
> I've hacked at the code a bit.
>
> Aaron.
>
So I was googling and I found this:
https://wiki.python.org/moin/PortingPythonToPy3k. Approach 1 might be a
good thing to try.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/