Mailing List Archive

On 07/07/14 11:14, Aaron Pelly wrote:
> There are also a couple of things I'd like to tweak.

For those that are interested, I have xmltv-proc-nz working with Python
3: https://github.com/apelly/xmltv-tools

At this stage it it's still a bit of a pain to set up. You will need to
install the tmdb & tvdb apis from my repo too. I have not ported any of
the installation or testing scripts for them. This may or may not cause
you a problem.

It's still early days, but advice and feedback would be appreciated.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

TL;DR Comments requested for a replacement xmltv grabber for NZ

I've been thinking about how to string all this together in an xmltv
friendly way.

My thoughts, based on my progress so far:
1) Change mhegepgsnoop to invent xmltv channel names based on somehting
like trim(name)+'.freeview.nz' and return all channel data, regardless
of which channels interest you. We have to decode it all anyway. This
means mhegepgsnoop no longer needs configuration. But will probably mean
renaming some of your channels in Myth. Can also make it handle utf-8
characters properly.

2) Process the data as per the xmltv spec, to exclude everything that is
not interesting. This should be easy enough to do by modifying an
existing grabber. The recommended new grabbers do seem more
feature-complete than our current one.

3) Within the modified grabber, call xmltv-proc-nz to augment any
remaining data. This processes the minimum amount of data, which is a
good thing.

I'm kind of reluctant to make yet another copy of an mheg reader,
knowing that my copy will surely fall into disrepair at some stage.
Perhaps the author would care to comment? On the other hand, the code
could really do with some comments and a bit of a clean up. Caching
would be good too, for testing. And it might be interesting to make it
Python 3 as well.

As for utf-8; I haven't had any to work with yet, because they have been
missing in the guide data. I am confident this can be made to work
properly with only minor changes to existing code.

I feel like I'm very close to something good here. xmltv-proc-nz
probably needs more options, but, from where I am, this should be
crazy-easy to implement.

Anyway, I'm kind of thinking out loud here, but if anyone has an opinion
I'm all ears.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 18/10/14 20:55, Aaron Pelly wrote:
> TL;DR Comments requested for a replacement xmltv grabber for NZ
>
> I've been thinking about how to string all this together in an xmltv
> friendly way.
>
> My thoughts, based on my progress so far:
> 1) Change mhegepgsnoop to invent xmltv channel names based on
> somehting like trim(name)+'.freeview.nz' and return all channel data,
> regardless of which channels interest you. We have to decode it all
> anyway. This means mhegepgsnoop no longer needs configuration. But
> will probably mean renaming some of your channels in Myth. Can also
> make it handle utf-8 characters properly.
>
> 2) Process the data as per the xmltv spec, to exclude everything that
> is not interesting. This should be easy enough to do by modifying an
> existing grabber. The recommended new grabbers do seem more
> feature-complete than our current one.
>
> 3) Within the modified grabber, call xmltv-proc-nz to augment any
> remaining data. This processes the minimum amount of data, which is a
> good thing.
>
> I'm kind of reluctant to make yet another copy of an mheg reader,
> knowing that my copy will surely fall into disrepair at some stage.
> Perhaps the author would care to comment? On the other hand, the code
> could really do with some comments and a bit of a clean up. Caching
> would be good too, for testing. And it might be interesting to make it
> Python 3 as well.
>
> As for utf-8; I haven't had any to work with yet, because they have
> been missing in the guide data. I am confident this can be made to
> work properly with only minor changes to existing code.
>
> I feel like I'm very close to something good here. xmltv-proc-nz
> probably needs more options, but, from where I am, this should be
> crazy-easy to implement.
>
> Anyway, I'm kind of thinking out loud here, but if anyone has an
> opinion I'm all ears.
>
Hi Aaron,

I'm the primary author of mhegepgsnoop. Others have made useful
contributions and I'm happy to accept changes but I have one
non-negotiable rule. Any changes must be backward compatible. So if you
want to _extend_ mhegepgsnoop in some way then go for it. Just add
another command line option and the help for that option. I'm happy to
help you with this.

I would strongly recommend against forking the project because I won't
support such a fork. Your comment about not making another mheg reader
is probably wiser than you think. Getting and parsing the mheg data was
by far the trickiest part of the code. So if it needs to change in
future there is quite a steep learning curve to acheive reasonable
understanding of this part of the code. But, hey, if you're up for the
challenge then go for it. :) Maybe you'll work out a better way to get
the data.

BTW if you like C (C++?) then have a look at the rb-download source to
see how it gets the mheg data.

Offer to clean up code and add more comments (where really useful)
gratefully accepted. :) Actually I know the code is a bit spaghetti.
Knowing what I know now I would write it much differently if I started
now. But it ain't broke so I don't fix it.

BTW I really don't see Python 3 compatibility as something that needs to
be tackled soon. Fine if you want to do it but it seems to me that
Python 2 will be around for a while yet. Just check out the number of
libs available for 2 vs 3. 2 won't go away until 3 has a similar number
of libs available.

Cheers
David

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 18/10/14 20:55, Aaron Pelly wrote:
> TL;DR Comments requested for a replacement xmltv grabber for NZ
>
> I've been thinking about how to string all this together in an xmltv
> friendly way.
>
> My thoughts, based on my progress so far:
> 1) Change mhegepgsnoop to invent xmltv channel names based on
> somehting like trim(name)+'.freeview.nz' and return all channel data,
> regardless of which channels interest you. We have to decode it all
> anyway. This means mhegepgsnoop no longer needs configuration. But
> will probably mean renaming some of your channels in Myth. Can also
> make it handle utf-8 characters properly.
Forgot to mention that mhegepgsnoop does support no config. It has the
option to use fuzzy string matching of channel names in the mheg epg vs
channel names in mythtv. So you just need to set up your myth channel
names similar to the mheg names. Use the verbose log output to see the
mheg names. Works for me.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 11:33, David Moore wrote:
> On 18/10/14 20:55, Aaron Pelly wrote:
>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>
>> I've been thinking about how to string all this together in an xmltv
>> friendly way.
>>
>> My thoughts, based on my progress so far:
>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>> somehting like trim(name)+'.freeview.nz' and return all channel data,
>> regardless of which channels interest you. We have to decode it all
>> anyway. This means mhegepgsnoop no longer needs configuration. But
>> will probably mean renaming some of your channels in Myth. Can also
>> make it handle utf-8 characters properly.
>>
>> 2) Process the data as per the xmltv spec, to exclude everything that
>> is not interesting. This should be easy enough to do by modifying an
>> existing grabber. The recommended new grabbers do seem more
>> feature-complete than our current one.
>>
>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>> remaining data. This processes the minimum amount of data, which is a
>> good thing.
>>
>> I'm kind of reluctant to make yet another copy of an mheg reader,
>> knowing that my copy will surely fall into disrepair at some stage.
>> Perhaps the author would care to comment? On the other hand, the code
>> could really do with some comments and a bit of a clean up. Caching
>> would be good too, for testing. And it might be interesting to make
>> it Python 3 as well.
>>
>> As for utf-8; I haven't had any to work with yet, because they have
>> been missing in the guide data. I am confident this can be made to
>> work properly with only minor changes to existing code.
>>
>> I feel like I'm very close to something good here. xmltv-proc-nz
>> probably needs more options, but, from where I am, this should be
>> crazy-easy to implement.
>>
>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>> opinion I'm all ears.
>>
> Hi Aaron,
>
> I'm the primary author of mhegepgsnoop. Others have made useful
> contributions and I'm happy to accept changes but I have one
> non-negotiable rule. Any changes must be backward compatible. So if
> you want to _extend_ mhegepgsnoop in some way then go for it. Just add
> another command line option and the help for that option.

Awesome! This is great news.

> Getting and parsing the mheg data was by far the trickiest part of the
> code. So if it needs to change in future there is quite a steep
> learning curve to acheive reasonable understanding of this part of the
> code.

Now the hard part has been done I might see what else is in there.

> But, hey, if you're up for the challenge then go for it. :) Maybe
> you'll work out a better way to get the data.

Ha!

> Offer to clean up code and add more comments (where really useful)
> gratefully accepted. :) Actually I know the code is a bit spaghetti.
> Knowing what I know now I would write it much differently if I started
> now.

My code rarely starts with a good plan, and I've been writing (not
Python!) for years. Hard problems like this are the best at causing
temporary code that "I'll get back to"

> it ain't broke so I don't fix it.

Good reasoning.

>
> BTW I really don't see Python 3 compatibility as something that needs
> to be tackled soon. Fine if you want to do it but it seems to me that
> Python 2 will be around for a while yet. Just check out the number of
> libs available for 2 vs 3. 2 won't go away until 3 has a similar
> number of libs available.

Yes, but I can't write Python, so I /really/ don't want to be able to
write Python 2 and Python 3.

One issue though: Do you see porting to Python 3 as breaking backwards
compatibility?

Anyway, as I said; it's early days. I'll have a better opinion once I've
hacked at the code a bit.

Aaron.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 13:08, Aaron Pelly wrote:
> On 19/10/14 11:33, David Moore wrote:
>> On 18/10/14 20:55, Aaron Pelly wrote:
>>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>>
>>> I've been thinking about how to string all this together in an xmltv
>>> friendly way.
>>>
>>> My thoughts, based on my progress so far:
>>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>>> somehting like trim(name)+'.freeview.nz' and return all channel
>>> data, regardless of which channels interest you. We have to decode
>>> it all anyway. This means mhegepgsnoop no longer needs
>>> configuration. But will probably mean renaming some of your channels
>>> in Myth. Can also make it handle utf-8 characters properly.
>>>
>>> 2) Process the data as per the xmltv spec, to exclude everything
>>> that is not interesting. This should be easy enough to do by
>>> modifying an existing grabber. The recommended new grabbers do seem
>>> more feature-complete than our current one.
>>>
>>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>>> remaining data. This processes the minimum amount of data, which is
>>> a good thing.
>>>
>>> I'm kind of reluctant to make yet another copy of an mheg reader,
>>> knowing that my copy will surely fall into disrepair at some stage.
>>> Perhaps the author would care to comment? On the other hand, the
>>> code could really do with some comments and a bit of a clean up.
>>> Caching would be good too, for testing. And it might be interesting
>>> to make it Python 3 as well.
>>>
>>> As for utf-8; I haven't had any to work with yet, because they have
>>> been missing in the guide data. I am confident this can be made to
>>> work properly with only minor changes to existing code.
>>>
>>> I feel like I'm very close to something good here. xmltv-proc-nz
>>> probably needs more options, but, from where I am, this should be
>>> crazy-easy to implement.
>>>
>>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>>> opinion I'm all ears.
>>>
>> Hi Aaron,
>>
>> I'm the primary author of mhegepgsnoop. Others have made useful
>> contributions and I'm happy to accept changes but I have one
>> non-negotiable rule. Any changes must be backward compatible. So if
>> you want to _extend_ mhegepgsnoop in some way then go for it. Just
>> add another command line option and the help for that option.
>
> Awesome! This is great news.
>
>> Getting and parsing the mheg data was by far the trickiest part of
>> the code. So if it needs to change in future there is quite a steep
>> learning curve to acheive reasonable understanding of this part of
>> the code.
>
> Now the hard part has been done I might see what else is in there.
>
>> But, hey, if you're up for the challenge then go for it. :) Maybe
>> you'll work out a better way to get the data.
>
> Ha!
>
>> Offer to clean up code and add more comments (where really useful)
>> gratefully accepted. :) Actually I know the code is a bit spaghetti.
>> Knowing what I know now I would write it much differently if I
>> started now.
>
> My code rarely starts with a good plan, and I've been writing (not
> Python!) for years. Hard problems like this are the best at causing
> temporary code that "I'll get back to"
>
>> it ain't broke so I don't fix it.
>
> Good reasoning.
>
>>
>> BTW I really don't see Python 3 compatibility as something that needs
>> to be tackled soon. Fine if you want to do it but it seems to me that
>> Python 2 will be around for a while yet. Just check out the number of
>> libs available for 2 vs 3. 2 won't go away until 3 has a similar
>> number of libs available.
>
> Yes, but I can't write Python, so I /really/ don't want to be able to
> write Python 2 and Python 3.
>
> One issue though: Do you see porting to Python 3 as breaking backwards
> compatibility?
>
> Anyway, as I said; it's early days. I'll have a better opinion once
> I've hacked at the code a bit.
>
> Aaron.
>
My understanding is that Python 3 is deliberately not backward
compatible with Python 2. So I guess it would be tedious to refactor the
code so it runs on both 2 and 3. I think I'm saying that support for 3
would be a fork of the 2 code and both would have to be maintained in
parallel for a while until everyone moves to 3.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 19/10/14 13:08, Aaron Pelly wrote:
> On 19/10/14 11:33, David Moore wrote:
>> On 18/10/14 20:55, Aaron Pelly wrote:
>>> TL;DR Comments requested for a replacement xmltv grabber for NZ
>>>
>>> I've been thinking about how to string all this together in an xmltv
>>> friendly way.
>>>
>>> My thoughts, based on my progress so far:
>>> 1) Change mhegepgsnoop to invent xmltv channel names based on
>>> somehting like trim(name)+'.freeview.nz' and return all channel
>>> data, regardless of which channels interest you. We have to decode
>>> it all anyway. This means mhegepgsnoop no longer needs
>>> configuration. But will probably mean renaming some of your channels
>>> in Myth. Can also make it handle utf-8 characters properly.
>>>
>>> 2) Process the data as per the xmltv spec, to exclude everything
>>> that is not interesting. This should be easy enough to do by
>>> modifying an existing grabber. The recommended new grabbers do seem
>>> more feature-complete than our current one.
>>>
>>> 3) Within the modified grabber, call xmltv-proc-nz to augment any
>>> remaining data. This processes the minimum amount of data, which is
>>> a good thing.
>>>
>>> I'm kind of reluctant to make yet another copy of an mheg reader,
>>> knowing that my copy will surely fall into disrepair at some stage.
>>> Perhaps the author would care to comment? On the other hand, the
>>> code could really do with some comments and a bit of a clean up.
>>> Caching would be good too, for testing. And it might be interesting
>>> to make it Python 3 as well.
>>>
>>> As for utf-8; I haven't had any to work with yet, because they have
>>> been missing in the guide data. I am confident this can be made to
>>> work properly with only minor changes to existing code.
>>>
>>> I feel like I'm very close to something good here. xmltv-proc-nz
>>> probably needs more options, but, from where I am, this should be
>>> crazy-easy to implement.
>>>
>>> Anyway, I'm kind of thinking out loud here, but if anyone has an
>>> opinion I'm all ears.
>>>
>> Hi Aaron,
>>
>> I'm the primary author of mhegepgsnoop. Others have made useful
>> contributions and I'm happy to accept changes but I have one
>> non-negotiable rule. Any changes must be backward compatible. So if
>> you want to _extend_ mhegepgsnoop in some way then go for it. Just
>> add another command line option and the help for that option.
>
> Awesome! This is great news.
>
>> Getting and parsing the mheg data was by far the trickiest part of
>> the code. So if it needs to change in future there is quite a steep
>> learning curve to acheive reasonable understanding of this part of
>> the code.
>
> Now the hard part has been done I might see what else is in there.
>
>> But, hey, if you're up for the challenge then go for it. :) Maybe
>> you'll work out a better way to get the data.
>
> Ha!
>
>> Offer to clean up code and add more comments (where really useful)
>> gratefully accepted. :) Actually I know the code is a bit spaghetti.
>> Knowing what I know now I would write it much differently if I
>> started now.
>
> My code rarely starts with a good plan, and I've been writing (not
> Python!) for years. Hard problems like this are the best at causing
> temporary code that "I'll get back to"
>
>> it ain't broke so I don't fix it.
>
> Good reasoning.
>
>>
>> BTW I really don't see Python 3 compatibility as something that needs
>> to be tackled soon. Fine if you want to do it but it seems to me that
>> Python 2 will be around for a while yet. Just check out the number of
>> libs available for 2 vs 3. 2 won't go away until 3 has a similar
>> number of libs available.
>
> Yes, but I can't write Python, so I /really/ don't want to be able to
> write Python 2 and Python 3.
>
> One issue though: Do you see porting to Python 3 as breaking backwards
> compatibility?
>
> Anyway, as I said; it's early days. I'll have a better opinion once
> I've hacked at the code a bit.
>
> Aaron.
>
So I was googling and I found this:
https://wiki.python.org/moin/PortingPythonToPy3k. Approach 1 might be a
good thing to try.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/