Mailing List Archive

epg.org.nz
Hi All

Is epg.org.nz ever going to get TV3+1 listings back?

What is the problem with this channel?


cheers

Paul


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
Paul wrote:
> Is epg.org.nz ever going to get TV3+1 listings back?
> What is the problem with this channel?


+1 channels are a complete and utter waste of bandwidth in this day and
age of timeshifting devices?



--
Criggie

http://criggie.org.nz/




_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
I see that I only have 3 days left too.

Robert Fisher

On Sat, 27 Jul 2019, 6:35 PM Adam Bennetts, <abennetts@gmail.com> wrote:

> Hi,
>
> Has the new data been moved over by any chance? Guide data hasn't been
> downloading for the last few days (since Tuesday maybe?). I had a
> chance to take a look today. On my box something isn't quite right, it
> seems tv_grab_nz-py 0.2.7 can't follow the redirect.
>
> I fixed by changing:
>
> if res.status != 200:
> return False
>
> to
>
> if res.status != 200:
> log.error('HTTP Status: %s' % res.status)
> return False
>
> to log HTTP errors and found the 301 redirect that was failing.
>
> And then this change:
>
> h = httplib.HTTPConnection(host)
>
> to
>
> h = httplib.HTTPSConnection(host)
>
> I then changed the source as I still seem to have a redirect issue
> with the original location (I checked the redirect itself and it's
> fine so my initial though that using HTTPSConnection would allow it to
> follow may not be right). Even if it doesn't resolve the redirect the
> change is needed.
>
> 'http://epg.nz/freeview.xml.gz',
>
> I also tried the new guide data at https://epg.nz/epg.xml and didn't
> see any obvious issues with it. There are differences to old data
> (subtitles?) that mean some recording rules duplicate checks needed to
> be changed but that's not a guide data issue.
>
> Thanks for the data, much appreciated.
>
> Cheers.
>
> On Tue, 16 Jul 2019 at 12:07, Hadley Rich <hads@nice.net.nz> wrote:
> >
> > Hi all,
> >
> > The box that generates the listings for epg.org.nz is going away in the
> > near future.
> >
> > That doesn't mean that the listings are going away, just the old box and
> > tuner card etc. that is used to capture the listings - it's pretty old
> now.
> >
> > I'm testing out a new version of listings that doesn't require any
> > physical hardware, you can test a sample here:
> >
> > https://epg.nz/epg.xml
> >
> > This will be replacing the existing data. I believe it's currently
> > missing a few fixups etc. that the existing data has which will be
> > worked on as time permits.
> >
> > If anyone wants to test it out and report any obvious errors that would
> > be appreciated.
> >
> > Once it's ready the old URL will be re-pointed so scripts should all
> > continue to work etc. with minimal interruption hopefully.
> >
> > Cheers,
> >
> > hads
> > --
> > http://nice.net.nz
> >
> > _______________________________________________
> > mythtvnz mailing list
> > mythtvnz@lists.ourshack.com
> > https://lists.ourshack.com/mailman/listinfo/mythtvnz
> > Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
Re: epg.org.nz [ In reply to ]
Hey Adam,

Thanks very much for the post that's great.

I'm not currently using the grabber so had forgotten to test that, seems
it's pretty old school and doesn't handle the redirect as you said.

I've relaxed the config on that site to allow http connections for now
until I can update the grabber to handle that better.

Feel free to let me know if there are more issues.

Cheers,

Hadley

On 27/07/19 6:33 PM, Adam Bennetts wrote:
> Hi,
>
> Has the new data been moved over by any chance? Guide data hasn't been
> downloading for the last few days (since Tuesday maybe?). I had a
> chance to take a look today. On my box something isn't quite right, it
> seems tv_grab_nz-py 0.2.7 can't follow the redirect.
>
> I fixed by changing:
>
> if res.status != 200:
> return False
>
> to
>
> if res.status != 200:
> log.error('HTTP Status: %s' % res.status)
> return False
>
> to log HTTP errors and found the 301 redirect that was failing.
>
> And then this change:
>
> h = httplib.HTTPConnection(host)
>
> to
>
> h = httplib.HTTPSConnection(host)
>
> I then changed the source as I still seem to have a redirect issue
> with the original location (I checked the redirect itself and it's
> fine so my initial though that using HTTPSConnection would allow it to
> follow may not be right). Even if it doesn't resolve the redirect the
> change is needed.
>
> 'http://epg.nz/freeview.xml.gz',
>
> I also tried the new guide data at https://epg.nz/epg.xml and didn't
> see any obvious issues with it. There are differences to old data
> (subtitles?) that mean some recording rules duplicate checks needed to
> be changed but that's not a guide data issue.
>
> Thanks for the data, much appreciated.
>
> Cheers.
>
> On Tue, 16 Jul 2019 at 12:07, Hadley Rich <hads@nice.net.nz> wrote:
>>
>> Hi all,
>>
>> The box that generates the listings for epg.org.nz is going away in the
>> near future.
>>
>> That doesn't mean that the listings are going away, just the old box and
>> tuner card etc. that is used to capture the listings - it's pretty old now.
>>
>> I'm testing out a new version of listings that doesn't require any
>> physical hardware, you can test a sample here:
>>
>> https://epg.nz/epg.xml
>>
>> This will be replacing the existing data. I believe it's currently
>> missing a few fixups etc. that the existing data has which will be
>> worked on as time permits.
>>
>> If anyone wants to test it out and report any obvious errors that would
>> be appreciated.
>>
>> Once it's ready the old URL will be re-pointed so scripts should all
>> continue to work etc. with minimal interruption hopefully.
>>
>> Cheers,
>>
>> hads
>> --
>> http://nice.net.nz
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
Thanks Hadley.

I just ran mythfilldatabase and now have 6 days of listings (previously 3)

Rob

On 28/07/19 9:24 AM, Hadley Rich wrote:
> Hey Adam,
>
> Thanks very much for the post that's great.
>
> I'm not currently using the grabber so had forgotten to test that,
> seems it's pretty old school and doesn't handle the redirect as you said.
>
> I've relaxed the config on that site to allow http connections for now
> until I can update the grabber to handle that better.
>
> Feel free to let me know if there are more issues.
>
> Cheers,
>
> Hadley
>
> On 27/07/19 6:33 PM, Adam Bennetts wrote:
>> Hi,
>>
>> Has the new data been moved over by any chance? Guide data hasn't been
>> downloading for the last few days (since Tuesday maybe?). I had a
>> chance to take a look today. On my box something isn't quite right, it
>> seems tv_grab_nz-py 0.2.7 can't follow the redirect.
>>
>> I fixed by changing:
>>
>>      if res.status != 200:
>>          return False
>>
>> to
>>
>>      if res.status != 200:
>>          log.error('HTTP Status: %s' % res.status)
>>          return False
>>
>> to log HTTP errors and found the 301 redirect that was failing.
>>
>> And then this change:
>>
>>      h = httplib.HTTPConnection(host)
>>
>> to
>>
>>      h = httplib.HTTPSConnection(host)
>>
>> I then changed the source as I still seem to have a redirect issue
>> with the original location (I checked the redirect itself and it's
>> fine so my initial though that using HTTPSConnection would allow it to
>> follow may not be right). Even if it doesn't resolve the redirect the
>> change is needed.
>>
>>       'http://epg.nz/freeview.xml.gz',
>>
>> I also tried the new guide data at https://epg.nz/epg.xml and didn't
>> see any obvious issues with it. There are differences to old data
>> (subtitles?) that mean some recording rules duplicate checks needed to
>> be changed but that's not a guide data issue.
>>
>> Thanks for the data, much appreciated.
>>
>> Cheers.
>>
>> On Tue, 16 Jul 2019 at 12:07, Hadley Rich <hads@nice.net.nz> wrote:
>>>
>>> Hi all,
>>>
>>> The box that generates the listings for epg.org.nz is going away in the
>>> near future.
>>>
>>> That doesn't mean that the listings are going away, just the old box
>>> and
>>> tuner card etc. that is used to capture the listings - it's pretty
>>> old now.
>>>
>>> I'm testing out a new version of listings that doesn't require any
>>> physical hardware, you can test a sample here:
>>>
>>> https://epg.nz/epg.xml
>>>
>>> This will be replacing the existing data. I believe it's currently
>>> missing a few fixups etc. that the existing data has which will be
>>> worked on as time permits.
>>>
>>> If anyone wants to test it out and report any obvious errors that would
>>> be appreciated.
>>>
>>> Once it's ready the old URL will be re-pointed so scripts should all
>>> continue to work etc. with minimal interruption hopefully.
>>>
>>> Cheers,
>>>
>>> hads
>>> --
>>> http://nice.net.nz
>>>
>>> _______________________________________________
>>> mythtvnz mailing list
>>> mythtvnz@lists.ourshack.com
>>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
Hi Hadley,
Thanks for your work with this.
It seems now though that my daily shows like the chase and TVNZ1 news are
seen as duplicates so show as already recorded.

Robert Fisher

On Sun, 28 Jul 2019, 9:25 AM Hadley Rich, <hads@nice.net.nz> wrote:

> Hey Adam,
>
> Thanks very much for the post that's great.
>
> I'm not currently using the grabber so had forgotten to test that, seems
> it's pretty old school and doesn't handle the redirect as you said.
>
> I've relaxed the config on that site to allow http connections for now
> until I can update the grabber to handle that better.
>
> Feel free to let me know if there are more issues.
>
> Cheers,
>
> Hadley
>
> On 27/07/19 6:33 PM, Adam Bennetts wrote:
> > Hi,
> >
> > Has the new data been moved over by any chance? Guide data hasn't been
> > downloading for the last few days (since Tuesday maybe?). I had a
> > chance to take a look today. On my box something isn't quite right, it
> > seems tv_grab_nz-py 0.2.7 can't follow the redirect.
> >
> > I fixed by changing:
> >
> > if res.status != 200:
> > return False
> >
> > to
> >
> > if res.status != 200:
> > log.error('HTTP Status: %s' % res.status)
> > return False
> >
> > to log HTTP errors and found the 301 redirect that was failing.
> >
> > And then this change:
> >
> > h = httplib.HTTPConnection(host)
> >
> > to
> >
> > h = httplib.HTTPSConnection(host)
> >
> > I then changed the source as I still seem to have a redirect issue
> > with the original location (I checked the redirect itself and it's
> > fine so my initial though that using HTTPSConnection would allow it to
> > follow may not be right). Even if it doesn't resolve the redirect the
> > change is needed.
> >
> > 'http://epg.nz/freeview.xml.gz',
> >
> > I also tried the new guide data at https://epg.nz/epg.xml and didn't
> > see any obvious issues with it. There are differences to old data
> > (subtitles?) that mean some recording rules duplicate checks needed to
> > be changed but that's not a guide data issue.
> >
> > Thanks for the data, much appreciated.
> >
> > Cheers.
> >
> > On Tue, 16 Jul 2019 at 12:07, Hadley Rich <hads@nice.net.nz> wrote:
> >>
> >> Hi all,
> >>
> >> The box that generates the listings for epg.org.nz is going away in the
> >> near future.
> >>
> >> That doesn't mean that the listings are going away, just the old box and
> >> tuner card etc. that is used to capture the listings - it's pretty old
> now.
> >>
> >> I'm testing out a new version of listings that doesn't require any
> >> physical hardware, you can test a sample here:
> >>
> >> https://epg.nz/epg.xml
> >>
> >> This will be replacing the existing data. I believe it's currently
> >> missing a few fixups etc. that the existing data has which will be
> >> worked on as time permits.
> >>
> >> If anyone wants to test it out and report any obvious errors that would
> >> be appreciated.
> >>
> >> Once it's ready the old URL will be re-pointed so scripts should all
> >> continue to work etc. with minimal interruption hopefully.
> >>
> >> Cheers,
> >>
> >> hads
> >> --
> >> http://nice.net.nz
> >>
> >> _______________________________________________
> >> mythtvnz mailing list
> >> mythtvnz@lists.ourshack.com
> >> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> >> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
> >
> > _______________________________________________
> > mythtvnz mailing list
> > mythtvnz@lists.ourshack.com
> > https://lists.ourshack.com/mailman/listinfo/mythtvnz
> > Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
> >
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
Re: epg.org.nz [ In reply to ]
On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:

>Hi Hadley,
>Thanks for your work with this.
>It seems now though that my daily shows like the chase and TVNZ1 news are
>seen as duplicates so show as already recorded.
>
>Robert Fisher

If the EPG source (TVNZ) is providing an ID for each episode, then
MythTV will be using that to decide whether they are duplicates.

I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
postprocessing using xmltv-proc-nz, pretty much the same way the
downloaded EPG is created. Here is an example of The Chase on TVNZ1
from my EPG:

<programme channel="tv1.freeviewnz.tv" start="20190806165500
+1200" stop="20190806180000 +1200">
<title>The Chase</title>
<desc>With thousands in cash at stake, four strangers
must go head to head with a ruthless quiz genius who?s determined to
send them home with nothing!</desc>
<episode-num system="dd_progid">10540106</episode-num>
<subtitles type="teletext" />
<rating system="Freeview">
<value>G</value>
</rating>
</programme>

It has a "dd_progid" of 10540106. In your downloaded EPG, are you
getting that? If you use the I key on a programme in the list of
recordings or in the Guide, you should see the same number in the
"Programme ID" field.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
The the same with "Newshub Live At 6pm". They are all marked as duplicates.

On Tue, Jul 30, 2019 at 6:23 PM Stephen Worthington <
stephen_agent@jsw.gen.nz> wrote:

> On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:
>
> >Hi Hadley,
> >Thanks for your work with this.
> >It seems now though that my daily shows like the chase and TVNZ1 news are
> >seen as duplicates so show as already recorded.
> >
> >Robert Fisher
>
> If the EPG source (TVNZ) is providing an ID for each episode, then
> MythTV will be using that to decide whether they are duplicates.
>
> I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
> postprocessing using xmltv-proc-nz, pretty much the same way the
> downloaded EPG is created. Here is an example of The Chase on TVNZ1
> from my EPG:
>
> <programme channel="tv1.freeviewnz.tv" start="20190806165500
> +1200" stop="20190806180000 +1200">
> <title>The Chase</title>
> <desc>With thousands in cash at stake, four strangers
> must go head to head with a ruthless quiz genius who’s determined to
> send them home with nothing!</desc>
> <episode-num system="dd_progid">10540106</episode-num>
> <subtitles type="teletext" />
> <rating system="Freeview">
> <value>G</value>
> </rating>
> </programme>
>
> It has a "dd_progid" of 10540106. In your downloaded EPG, are you
> getting that? If you use the I key on a programme in the list of
> recordings or in the Guide, you should see the same number in the
> "Programme ID" field.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
Re: epg.org.nz [ In reply to ]
Hi,

Yep, the episode-num tag is missing from the new data. I didn't have
an older copy when I was looking at issues on the weekend and missed
that it was this causing the recording rule issues I noticed. I hunted
around and found an older copy of the XML and, as expected, it did
have an episode-num tag set properly.

Easy fix for now is to turn off duplicate checking by setting the
duplicate check method to none for each rule impacted by this.

Hopefully Hadley can find where that info went missing and add it back in.

Cheers.


On Tue, 30 Jul 2019 at 18:23, Stephen Worthington
<stephen_agent@jsw.gen.nz> wrote:
>
> On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:
>
> >Hi Hadley,
> >Thanks for your work with this.
> >It seems now though that my daily shows like the chase and TVNZ1 news are
> >seen as duplicates so show as already recorded.
> >
> >Robert Fisher
>
> If the EPG source (TVNZ) is providing an ID for each episode, then
> MythTV will be using that to decide whether they are duplicates.
>
> I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
> postprocessing using xmltv-proc-nz, pretty much the same way the
> downloaded EPG is created. Here is an example of The Chase on TVNZ1
> from my EPG:
>
> <programme channel="tv1.freeviewnz.tv" start="20190806165500
> +1200" stop="20190806180000 +1200">
> <title>The Chase</title>
> <desc>With thousands in cash at stake, four strangers
> must go head to head with a ruthless quiz genius who’s determined to
> send them home with nothing!</desc>
> <episode-num system="dd_progid">10540106</episode-num>
> <subtitles type="teletext" />
> <rating system="Freeview">
> <value>G</value>
> </rating>
> </programme>
>
> It has a "dd_progid" of 10540106. In your downloaded EPG, are you
> getting that? If you use the I key on a programme in the list of
> recordings or in the Guide, you should see the same number in the
> "Programme ID" field.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
Hi guys,

Thanks for the heads up I thought episode-num was in there. I'll look
into this it soon as I can, I'm traveling right now so hopefully in the
next evening or two, otherwise the weekend.

Cheers,

Hadley

On 30/07/19 5:32 PM, Adam Bennetts wrote:
> Hi,
>
> Yep, the episode-num tag is missing from the new data. I didn't have
> an older copy when I was looking at issues on the weekend and missed
> that it was this causing the recording rule issues I noticed. I hunted
> around and found an older copy of the XML and, as expected, it did
> have an episode-num tag set properly.
>
> Easy fix for now is to turn off duplicate checking by setting the
> duplicate check method to none for each rule impacted by this.
>
> Hopefully Hadley can find where that info went missing and add it back in.
>
> Cheers.
>
>
> On Tue, 30 Jul 2019 at 18:23, Stephen Worthington
> <stephen_agent@jsw.gen.nz> wrote:
>>
>> On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:
>>
>>> Hi Hadley,
>>> Thanks for your work with this.
>>> It seems now though that my daily shows like the chase and TVNZ1 news are
>>> seen as duplicates so show as already recorded.
>>>
>>> Robert Fisher
>>
>> If the EPG source (TVNZ) is providing an ID for each episode, then
>> MythTV will be using that to decide whether they are duplicates.
>>
>> I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
>> postprocessing using xmltv-proc-nz, pretty much the same way the
>> downloaded EPG is created. Here is an example of The Chase on TVNZ1
>> from my EPG:
>>
>> <programme channel="tv1.freeviewnz.tv" start="20190806165500
>> +1200" stop="20190806180000 +1200">
>> <title>The Chase</title>
>> <desc>With thousands in cash at stake, four strangers
>> must go head to head with a ruthless quiz genius who’s determined to
>> send them home with nothing!</desc>
>> <episode-num system="dd_progid">10540106</episode-num>
>> <subtitles type="teletext" />
>> <rating system="Freeview">
>> <value>G</value>
>> </rating>
>> </programme>
>>
>> It has a "dd_progid" of 10540106. In your downloaded EPG, are you
>> getting that? If you use the I key on a programme in the list of
>> recordings or in the Guide, you should see the same number in the
>> "Programme ID" field.
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
I've just looked at this and it seems that episode-num does in fact
appear to be in there:

795 > > <title>The Block NZ: Firehouse (encore)</title>
796 > > <sub-title>The Block NZ: Firehouse</sub-title>
797 > > <sub-title>New Episode</sub-title>
798 > > <desc>The teams embark on their biggest space yet and push
the boundaries on design. Plus it's the return of Block Stars! S8
Ep29</desc>
799 > > <date>2019</date>
800 > > <episode-num system="xmltv_ns">7.28.0</episode-num>
801 > > <video>
802 > > > <quality>HDTV</quality>
803 > > </video>
804 > > <previously-shown />
805 > > <subtitles type="teletext" />


On 30/07/19 8:47 PM, Hadley Rich wrote:
> Hi guys,
>
> Thanks for the heads up I thought episode-num was in there. I'll look
> into this it soon as I can, I'm traveling right now so hopefully in the
> next evening or two, otherwise the weekend.
>
> Cheers,
>
> Hadley
>
> On 30/07/19 5:32 PM, Adam Bennetts wrote:
>> Hi,
>>
>> Yep, the episode-num tag is missing from the new data. I didn't have
>> an older copy when I was looking at issues on the weekend and missed
>> that it was this causing the recording rule issues I noticed. I hunted
>> around and found an older copy of the XML and, as expected, it did
>> have an episode-num tag set properly.
>>
>> Easy fix for now is to turn off duplicate checking by setting the
>> duplicate check method to none for each rule impacted by this.
>>
>> Hopefully Hadley can find where that info went missing and add it back
>> in.
>>
>> Cheers.
>>
>>
>> On Tue, 30 Jul 2019 at 18:23, Stephen Worthington
>> <stephen_agent@jsw.gen.nz> wrote:
>>>
>>> On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:
>>>
>>>> Hi Hadley,
>>>> Thanks for your work with this.
>>>> It seems now though that my daily shows like the chase and TVNZ1
>>>> news are
>>>> seen as duplicates so show as already recorded.
>>>>
>>>> Robert Fisher
>>>
>>> If the EPG source (TVNZ) is providing an ID for each episode, then
>>> MythTV will be using that to decide whether they are duplicates.
>>>
>>> I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
>>> postprocessing using xmltv-proc-nz, pretty much the same way the
>>> downloaded EPG is created.  Here is an example of The Chase on TVNZ1
>>> from my EPG:
>>>
>>>          <programme channel="tv1.freeviewnz.tv" start="20190806165500
>>> +1200" stop="20190806180000 +1200">
>>>                  <title>The Chase</title>
>>>                  <desc>With thousands in cash at stake, four strangers
>>> must go head to head with a ruthless quiz genius who’s determined to
>>> send them home with nothing!</desc>
>>>                  <episode-num system="dd_progid">10540106</episode-num>
>>>                  <subtitles type="teletext" />
>>>                  <rating system="Freeview">
>>>                          <value>G</value>
>>>                  </rating>
>>>          </programme>
>>>
>>> It has a "dd_progid" of 10540106.  In your downloaded EPG, are you
>>> getting that?  If you use the I key on a programme in the list of
>>> recordings or in the Guide, you should see the same number in the
>>> "Programme ID" field.
>>>
>>> _______________________________________________
>>> mythtvnz mailing list
>>> mythtvnz@lists.ourshack.com
>>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
I'm no expert but what I see (in mythweb) the options for duplicates
only refer to subtitles and description so I assume they are no longer
included in the data.

Rob

On 4/08/19 9:42 AM, Hadley Rich wrote:
> I've just looked at this and it seems that episode-num does in fact
> appear to be in there:
>
>   795 > > <title>The Block NZ: Firehouse (encore)</title>
>   796 > > <sub-title>The Block NZ: Firehouse</sub-title>
>   797 > > <sub-title>New Episode</sub-title>
>   798 > > <desc>The teams embark on their biggest space yet and push
> the boundaries on design. Plus it's the return of Block Stars! S8
> Ep29</desc>
>   799 > > <date>2019</date>
>   800 > > <episode-num system="xmltv_ns">7.28.0</episode-num>
>   801 > > <video>
>   802 > > > <quality>HDTV</quality>
>   803 > > </video>
>   804 > > <previously-shown />
>   805 > > <subtitles type="teletext" />
>
>
> On 30/07/19 8:47 PM, Hadley Rich wrote:
>> Hi guys,
>>
>> Thanks for the heads up I thought episode-num was in there. I'll look
>> into this it soon as I can, I'm traveling right now so hopefully in
>> the next evening or two, otherwise the weekend.
>>
>> Cheers,
>>
>> Hadley
>>
>> On 30/07/19 5:32 PM, Adam Bennetts wrote:
>>> Hi,
>>>
>>> Yep, the episode-num tag is missing from the new data. I didn't have
>>> an older copy when I was looking at issues on the weekend and missed
>>> that it was this causing the recording rule issues I noticed. I hunted
>>> around and found an older copy of the XML and, as expected, it did
>>> have an episode-num tag set properly.
>>>
>>> Easy fix for now is to turn off duplicate checking by setting the
>>> duplicate check method to none for each rule impacted by this.
>>>
>>> Hopefully Hadley can find where that info went missing and add it
>>> back in.
>>>
>>> Cheers.
>>>
>>>
>>> On Tue, 30 Jul 2019 at 18:23, Stephen Worthington
>>> <stephen_agent@jsw.gen.nz> wrote:
>>>>
>>>> On Tue, 30 Jul 2019 15:50:45 +1200, you wrote:
>>>>
>>>>> Hi Hadley,
>>>>> Thanks for your work with this.
>>>>> It seems now though that my daily shows like the chase and TVNZ1
>>>>> news are
>>>>> seen as duplicates so show as already recorded.
>>>>>
>>>>> Robert Fisher
>>>>
>>>> If the EPG source (TVNZ) is providing an ID for each episode, then
>>>> MythTV will be using that to decide whether they are duplicates.
>>>>
>>>> I get my EPG from the broadcast MHEG5 data using mhegepgsnoop, and do
>>>> postprocessing using xmltv-proc-nz, pretty much the same way the
>>>> downloaded EPG is created.  Here is an example of The Chase on TVNZ1
>>>> from my EPG:
>>>>
>>>>          <programme channel="tv1.freeviewnz.tv" start="20190806165500
>>>> +1200" stop="20190806180000 +1200">
>>>>                  <title>The Chase</title>
>>>>                  <desc>With thousands in cash at stake, four strangers
>>>> must go head to head with a ruthless quiz genius who’s determined to
>>>> send them home with nothing!</desc>
>>>>                  <episode-num
>>>> system="dd_progid">10540106</episode-num>
>>>>                  <subtitles type="teletext" />
>>>>                  <rating system="Freeview">
>>>>                          <value>G</value>
>>>>                  </rating>
>>>>          </programme>
>>>>
>>>> It has a "dd_progid" of 10540106.  In your downloaded EPG, are you
>>>> getting that?  If you use the I key on a programme in the list of
>>>> recordings or in the Guide, you should see the same number in the
>>>> "Programme ID" field.
>>>>
>>>> _______________________________________________
>>>> mythtvnz mailing list
>>>> mythtvnz@lists.ourshack.com
>>>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>>>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>>
>>> _______________________________________________
>>> mythtvnz mailing list
>>> mythtvnz@lists.ourshack.com
>>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>>>
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
On 5/8/19 7:40 am, Robert Fisher wrote:
> I'm no expert but what I see (in mythweb) the options for duplicates
> only refer to subtitles and description so I assume they are no longer
> included in the data.

Those are definitely in there.

I did just notice that sub-title could be added twice due to the new
data processing script adding it as well as it being added by
xmltv-proc-nz so I've fixed that in case it helps.

--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
The problem (for me at least) is that the subtitles and description are
the same for every showing of these daily shows so they are seen as
duplicates.

The work-around (Duplicate Check Method to: None) works but can be a
problem when, for example, recordings are repeated on the +1 channels
and using a tuner unnecessarily.

Rob.

On 5/08/19 9:14 AM, Hadley Rich wrote:
> On 5/8/19 7:40 am, Robert Fisher wrote:
>> I'm no expert but what I see (in mythweb) the options for duplicates
>> only refer to subtitles and description so I assume they are no
>> longer included in the data.
>
> Those are definitely in there.
>
> I did just notice that sub-title could be added twice due to the new
> data processing script adding it as well as it being added by
> xmltv-proc-nz so I've fixed that in case it helps.
>

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
On 04/08/19 09:42, Hadley Rich wrote:
> I've just looked at this and it seems that episode-num does in fact
> appear to be in there:
>
>   795 > > <title>The Block NZ: Firehouse (encore)</title>
>   796 > > <sub-title>The Block NZ: Firehouse</sub-title>
>   797 > > <sub-title>New Episode</sub-title>
>   798 > > <desc>The teams embark on their biggest space yet and push the
> boundaries on design. Plus it's the return of Block Stars! S8 Ep29</desc>
>   799 > > <date>2019</date>
>   800 > > <episode-num system="xmltv_ns">7.28.0</episode-num>
>   801 > > <video>
>   802 > > > <quality>HDTV</quality>
>   803 > > </video>
>   804 > > <previously-shown />
>   805 > > <subtitles type="teletext" />
>
>

Hads,

The problem occurs with programmes that do not have a meaningful
series/episode identification e.g. "1 News at 6pm".

I have looked back through my xmltv archive and here are the epg xml
records for 7 August 2018 and 7 August 2019 for "1 News at 6 pm".

epg_20180807_030125.xml


<title>1 News At 6pm</title>
<desc>The nation's leading team of journalists brings
viewers the latest news and sport, plus the most comprehensive w
eather report.</desc>
<episode-num system="dd_progid">10506097</episode-num>
<subtitles type="teletext" />
</programme>





epg_20190807_030107.xml


<title>1 News At 6pm</title>
<sub-title>1 News At 6pm</sub-title>
<desc>The nation's leading team of journalists brings
viewers the latest news and sport, plus the most comprehensive w
eather report.</desc>
<date>2019</date>
<subtitles type="teletext" />
</programme>

Note that the last time the "episode-num" term appeared in the epg file
for "1 News at 6pm" was the file I downloaded at approx 3:00 am on 7/8/2019.

I wonder if you have somehow switched your "snoop" system from mheg to
Sky for the Freeview channels?

Regards

Tony Sauri



_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
On 7/8/19 7:40 am, Tony Sauri wrote:
> The problem occurs with programmes that do not have a meaningful
> series/episode identification e.g. "1 News at 6pm".
>
> I have looked back through my xmltv archive and here are the epg xml
> records for 7 August 2018 and 7 August 2019 for "1 News at 6 pm".

[...]

> Note that the last time the "episode-num" term appeared in the epg file
> for "1 News at 6pm" was the file I downloaded at approx 3:00 am on 7/8/2019.

Hey Tony,

Perfect, thanks for the info, that's allowed me to make a small change
which will hopefully assist MythTV match those duplicates and not
regress the data for anything else.

> I wonder if you have somehow switched your "snoop" system from mheg to
> Sky for the Freeview channels?

I've not had anything to do with Sky for years :D

The new source has more data and is easier to process, it's just a
matter of my time to pull it all through and make that available in a
suitable format.

Keep the suggestions coming people.

Cheers,

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
Love your work Hadley.

On 7/08/19 9:52 AM, Hadley Rich wrote:
> On 7/8/19 7:40 am, Tony Sauri wrote:
>> The problem occurs with programmes that do not have a meaningful
>> series/episode identification e.g. "1 News at 6pm".
>>
>> I have looked back through my xmltv archive and here are the epg xml
>> records for  7 August 2018 and 7 August 2019 for "1 News at 6 pm".
>
> [...]
>
>> Note that the last time the "episode-num" term appeared in the epg file
>> for "1 News at 6pm" was the file I downloaded at approx 3:00 am on
>> 7/8/2019.
>
> Hey Tony,
>
> Perfect, thanks for the info, that's allowed me to make a small change
> which will hopefully assist MythTV match those duplicates and not
> regress the data for anything else.
>
>> I wonder if you have somehow switched your "snoop" system from mheg to
>> Sky for the Freeview channels?
>
> I've not had anything to do with Sky for years :D
>
> The new source has more data and is easier to process, it's just a
> matter of my time to pull it all through and make that available in a
> suitable format.
>
> Keep the suggestions coming people.
>
> Cheers,
>
> hads

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
On 07/08/19 09:52, Hadley Rich wrote:

>
> that's allowed me to make a small change
> which will hopefully assist MythTV match those duplicates and not
> regress the data for anything else.
suitable format.
>
> Keep the suggestions coming people.
>

Hads,

I see you have already made the amendment to the file

However there is a possible small problem to be addressed.

This problem may be specific to how I record programmes.

I set the record at any time or any channel on all of my schedule rules.
I have also given any +1 channels a higher recording priority because
the recordings are smaller so use less disk space, I cant see any
difference between 1080i and 765p on my system, and the scheduler get
more flexibility to record programmes on choice that screens some
programmes multiple times a week.

It seems that the cases that bring the +1 channels into play still do
not record after your latest change. This can be worked around by
deleting the existing recording schedule and then re-entering it again.

I have not got to the bottom of the problem although I think it is to do
with the records in the oldrecorded SQL table.

I will be having another look tomorrow in the hope of finding a better
solution.

Tony

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.ourshack.com
https://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: epg.org.nz [ In reply to ]
I have just realised that the "workaround" mentioned previously (setting
duplicate method to "none") is not the best.

If duplicate method is simply set to "subtitle" (not "subtitle and
description" as before) then it works fine.

Robert Fisher

On Mon, 5 Aug 2019, 11:06 AM Robert Fisher, <robert@fisher.net.nz> wrote:

> The problem (for me at least) is that the subtitles and description are
> the same for every showing of these daily shows so they are seen as
> duplicates.
>
> The work-around (Duplicate Check Method to: None) works but can be a
> problem when, for example, recordings are repeated on the +1 channels
> and using a tuner unnecessarily.
>
> Rob.
>
> On 5/08/19 9:14 AM, Hadley Rich wrote:
> > On 5/8/19 7:40 am, Robert Fisher wrote:
> >> I'm no expert but what I see (in mythweb) the options for duplicates
> >> only refer to subtitles and description so I assume they are no
> >> longer included in the data.
> >
> > Those are definitely in there.
> >
> > I did just notice that sub-title could be added twice due to the new
> > data processing script adding it as well as it being added by
> > xmltv-proc-nz so I've fixed that in case it helps.
> >
>