Mailing List Archive

Fwd: Fwd: Re: channel4 hd has no eit - cross-eit
I use 65692 which does work but the number may differ for you, test the
SD channel you pick as source for the EIT of the HD channel.

I am testing the following to cope with the clocks changing.

I know its cringe worthy but might work, the two versions of crosseit
have the single line change.

?# Winter --
??? # date = dateobj # + datetime.timedelta(seconds=time.altzone)
??? # Summer
??? date = dateobj + datetime.timedelta(seconds=time.altzone)

?#!/bin/bash
if date +%Z | grep -e BST; then
??? echo "Summer"
??? ./cross-eit3S.py 65692,"hd.channel4.com"
else
??? echo "Winter"
??? ./cross-eit3W.py 65692,"hd.channel4.com"
fi
/usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
--sourceid 1 --xmlfile output.xml



-------- Forwarded Message --------
Subject: Re: [mythtv-users] Fwd: Re: channel4 hd has no eit - cross-eit
Date: Sun, 5 Sep 2021 11:00:43 +0100
From: John <jksjdevelop@gmail.com>
To: mythtv-users@mythtv.org



Ok you had me going there. Ignore previous statements of mine!

The Channel to copy from is Channel 4 which does have listings

So select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid
from channel where callsign like("Channel 4");

+-----------+--------+---------+----------+-----------+---------------------+---------+
| serviceid | chanid | channum | sourceid | callsign? |
deleted???????????? | xmltvid |
+-----------+--------+---------+----------+-----------+---------------------+---------+
|????? 9211 |? 19211 | 60001?? |??????? 0 | Channel 4 | 2020-11-19
12:25:08 |???????? |
|????? 9211 |? 65692 | 113???? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
|????? 9212 |? 65693 | 60024?? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
|????? 9213 |? 65694 | 60025?? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
|????? 9214 |? 65695 | 60026?? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
|????? 9216 |? 65696 | 60027?? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
|????? 8320 |? 65827 | 60150?? |??????? 1 | Channel 4 |
NULL??????????????? |???????? |
+-----------+--------+---------+----------+-----------+---------------------+---------+
7 rows in set (0.001 sec)
So the chanid to use is the undeleted ones of the above.

I see I have your choice of 19211 marked as deleted so its probably no
longer used.
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:

> I use 65692 which does work but the number may differ for you, test the SD
> channel you pick as source for the EIT of the HD channel.
>
How can I tell which one it is? I see no clue from the query below as to
which of them is being used for EIT.

select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
channel where callsign like("%Channel 4%");

+-----------+--------+---------+----------+--------------+---------------------+-----------------+

| serviceid | chanid | channum | sourceid | callsign | deleted
| xmltvid |

+-----------+--------+---------+----------+--------------+---------------------+-----------------+

| 9211 | 10211 | 9211 | 1 | Channel 4 | 2021-04-13
22:25:17 | |

| 8311 | 9311 | 8311 | 1 | Channel 4+1 | 2021-04-13
22:25:17 | |

| 8316 | 9316 | 8316 | 1 | Channel 4+1 | 2021-04-13
22:25:17 | |

| 8320 | 9320 | 8320 | 1 | Channel 4 | 2021-04-13
22:25:17 | |

| 21200 | 22200 | 21200 | 1 | Channel 4 HD | 2021-04-13
22:25:17 | hd.channel4.com |

| 9211 | 19211 | 9211 | 1 | Channel 4 | NULL
| |

| 9212 | 19212 | 9212 | 1 | Channel 4 | NULL
| |

| 9213 | 19213 | 9213 | 1 | Channel 4 | NULL
| |

| 9214 | 19214 | 9214 | 1 | Channel 4 | NULL
| |

| 9216 | 19216 | 9216 | 1 | Channel 4 | NULL
| |

| 8311 | 18311 | 8311 | 1 | Channel 4+1 | NULL
| |

| 8312 | 18312 | 8312 | 1 | Channel 4+1 | NULL
| |

| 8314 | 18314 | 8314 | 1 | Channel 4+1 | NULL
| |

| 8316 | 18316 | 8316 | 1 | Channel 4+1 | NULL
| |

| 8320 | 18320 | 8320 | 1 | Channel 4 | NULL
| |

| 21200 | 31200 | 21200 | 1 | Channel 4 HD | NULL
| hd.channel4.com |

| 22620 | 32620 | 22620 | 1 | Channel 44 | NULL
| |

+-----------+--------+---------+----------+--------------+---------------------+-----------------+

17 rows in set (0.01 sec)


I also tried looking at all the existing column headings in "channel" (with
"select * from channel") but none suggested that a channel was being used
for EIT. Obviously that info must be in some other table. Would you know
which?



> I am testing the following to cope with the clocks changing.
>
> I know its cringe worthy but might work, the two versions of crosseit have
> the single line change.
>
> # Winter --
> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
> # Summer
> date = dateobj + datetime.timedelta(seconds=time.altzone)
>
> #!/bin/bash
> if date +%Z | grep -e BST; then
> echo "Summer"
> ./cross-eit3S.py 65692,"hd.channel4.com"
> else
> echo "Winter"
> ./cross-eit3W.py 65692,"hd.channel4.com"
> fi
> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
> --sourceid 1 --xmlfile output.xml
>

You are right that the mix of bash and python is somewhat cringe-worthy: it
would be neater to do the change in python rather than calling two versions
of the script.

However a more serious problem is that this approach changes the time zone
for all the dates in the EIT for that week. In steady state that's likely
to be fine, but in those two transition weeks of the year, when one batch
of listings has some listings before the change and some after the change,
some of those listings will be assigned the wrong offset.

The correct but slightly more laborious fix, in my opinion, would be to
apply the offset to each individual programme listing in the batch, thus
possibly having programmes in the same batch with different UTC offsets,
which will happen during those transition weeks. I'll code it up in python
when I have a moment, starting from the code that Stephen posted yesterday
that got rid of the absurdly long lines.
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Mon, 6 Sep 2021 07:01:01 +0100
UB40D via mythtv-users <mythtv-users@mythtv.org> wrote:

> On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
>
> > I use 65692 which does work but the number may differ for you, test
> > the SD channel you pick as source for the EIT of the HD channel.
> >
> How can I tell which one it is? I see no clue from the query below as
> to which of them is being used for EIT.
>
> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid
> from channel where callsign like("%Channel 4%");
>
> +-----------+--------+---------+----------+--------------+---------------------+-----------------+
>
> | serviceid | chanid | channum | sourceid | callsign | deleted
> | xmltvid |
>
> +-----------+--------+---------+----------+--------------+---------------------+-----------------+
>
> | 9211 | 10211 | 9211 | 1 | Channel 4 | 2021-04-13
> 22:25:17 | |
>
> | 8311 | 9311 | 8311 | 1 | Channel 4+1 | 2021-04-13
> 22:25:17 | |
>
> | 8316 | 9316 | 8316 | 1 | Channel 4+1 | 2021-04-13
> 22:25:17 | |
>
> | 8320 | 9320 | 8320 | 1 | Channel 4 | 2021-04-13
> 22:25:17 | |
>
> | 21200 | 22200 | 21200 | 1 | Channel 4 HD | 2021-04-13
> 22:25:17 | hd.channel4.com |
>
> | 9211 | 19211 | 9211 | 1 | Channel 4 | NULL
> | |
>
> | 9212 | 19212 | 9212 | 1 | Channel 4 | NULL
> | |
>
> | 9213 | 19213 | 9213 | 1 | Channel 4 | NULL
> | |
>
> | 9214 | 19214 | 9214 | 1 | Channel 4 | NULL
> | |
>
> | 9216 | 19216 | 9216 | 1 | Channel 4 | NULL
> | |
>
> | 8311 | 18311 | 8311 | 1 | Channel 4+1 | NULL
> | |
>
> | 8312 | 18312 | 8312 | 1 | Channel 4+1 | NULL
> | |
>
> | 8314 | 18314 | 8314 | 1 | Channel 4+1 | NULL
> | |
>
> | 8316 | 18316 | 8316 | 1 | Channel 4+1 | NULL
> | |
>
> | 8320 | 18320 | 8320 | 1 | Channel 4 | NULL
> | |
>
> | 21200 | 31200 | 21200 | 1 | Channel 4 HD | NULL
> | hd.channel4.com |
>
> | 22620 | 32620 | 22620 | 1 | Channel 44 | NULL
> | |
>
> +-----------+--------+---------+----------+--------------+---------------------+-----------------+
>
> 17 rows in set (0.01 sec)
>
>
> I also tried looking at all the existing column headings in "channel"
> (with "select * from channel") but none suggested that a channel was
> being used for EIT. Obviously that info must be in some other table.
> Would you know which?
>
>

There is a field called useonairguide in the channel table. When set to
1 the EIT data is used for that channel. See
https://www.mythtv.org/wiki/EIT

Steve
--
____________________________________________________________________
Steve Evans E-mail: mailto:stevee@gorbag.com
Registered Linux user #217906: http://counter.li.org
Public Encryption Key: http://www.gorbag.com/public-key.html
____________________________________________________________________

5.10.61-gentoo Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz GNU/Linux

10:33:57 up 5 days, 23:10, 5 users, load average: 0.09, 0.26, 0.25

He who hates vices hates mankind.
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Mon, 6 Sep 2021 07:01:01 +0100, you wrote:

>On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
>
>> I use 65692 which does work but the number may differ for you, test the SD
>> channel you pick as source for the EIT of the HD channel.
>>
>How can I tell which one it is? I see no clue from the query below as to
>which of them is being used for EIT.
>
> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
>channel where callsign like("%Channel 4%");
[snip]
>
>I also tried looking at all the existing column headings in "channel" (with
>"select * from channel") but none suggested that a channel was being used
>for EIT. Obviously that info must be in some other table. Would you know
>which?

The field you want is "useonairguide".

>> I am testing the following to cope with the clocks changing.
>>
>> I know its cringe worthy but might work, the two versions of crosseit have
>> the single line change.
>>
>> # Winter --
>> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
>> # Summer
>> date = dateobj + datetime.timedelta(seconds=time.altzone)
>>
>> #!/bin/bash
>> if date +%Z | grep -e BST; then
>> echo "Summer"
>> ./cross-eit3S.py 65692,"hd.channel4.com"
>> else
>> echo "Winter"
>> ./cross-eit3W.py 65692,"hd.channel4.com"
>> fi
>> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
>> --sourceid 1 --xmlfile output.xml
>>
>
>You are right that the mix of bash and python is somewhat cringe-worthy: it
>would be neater to do the change in python rather than calling two versions
>of the script.
>
>However a more serious problem is that this approach changes the time zone
>for all the dates in the EIT for that week. In steady state that's likely
>to be fine, but in those two transition weeks of the year, when one batch
>of listings has some listings before the change and some after the change,
>some of those listings will be assigned the wrong offset.
>
>The correct but slightly more laborious fix, in my opinion, would be to
>apply the offset to each individual programme listing in the batch, thus
>possibly having programmes in the same batch with different UTC offsets,
>which will happen during those transition weeks. I'll code it up in python
>when I have a moment, starting from the code that Stephen posted yesterday
>that got rid of the absurdly long lines.

No, there will not be different offsets. Cross-eit.py is exporting
data from the MythTV database. MythTV internally only works in UTC,
so all the exported data will be UTC. So all that needs to be done is
to add the string " +0000" on the end of the "start=" and "stop="
timestamps.

When mythfilldatabase reads an XMLTV file, it will use the timezone
offsets to convert each timestamp to UTC before it puts it into the
database. EPG collected from a TV channel (using EIT, MHEG5 or OpenTV
which are the formats I have used) is usually in local time and
therefore will have the correct offsets on it. When that collected
data crosses a daylight saving change, you will see the offset values
change (unless the EPG provider has screwed things up, which is fairly
common in this part of the world).

It is possible that the old cross-eit.py code that included using
timedelta was from when MythTV used local times in its database. That
caused hideous complications in the code for handling daylight saving,
so the internal times were all changed to UTC many versions of MythTV
ago now.

I have put an updated version of cross-eit.py (v2.1) with this change
on my web server:

http://www.jsw.gen.nz/mythtv/cross-eit.py
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On 06/09/2021 10:46, Stephen Worthington wrote:
> On Mon, 6 Sep 2021 07:01:01 +0100, you wrote:
>
>> On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
>>
>>> I use 65692 which does work but the number may differ for you, test the SD
>>> channel you pick as source for the EIT of the HD channel.
>>>
>> How can I tell which one it is? I see no clue from the query below as to
>> which of them is being used for EIT.
>>
>> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
>> channel where callsign like("%Channel 4%");
> [snip]
>> I also tried looking at all the existing column headings in "channel" (with
>> "select * from channel") but none suggested that a channel was being used
>> for EIT. Obviously that info must be in some other table. Would you know
>> which?
> The field you want is "useonairguide".
>
>>> I am testing the following to cope with the clocks changing.
>>>
>>> I know its cringe worthy but might work, the two versions of crosseit have
>>> the single line change.
>>>
>>> # Winter --
>>> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
>>> # Summer
>>> date = dateobj + datetime.timedelta(seconds=time.altzone)
>>>
>>> #!/bin/bash
>>> if date +%Z | grep -e BST; then
>>> echo "Summer"
>>> ./cross-eit3S.py 65692,"hd.channel4.com"
>>> else
>>> echo "Winter"
>>> ./cross-eit3W.py 65692,"hd.channel4.com"
>>> fi
>>> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
>>> --sourceid 1 --xmlfile output.xml
>>>
>> You are right that the mix of bash and python is somewhat cringe-worthy: it
>> would be neater to do the change in python rather than calling two versions
>> of the script.
>>
>> However a more serious problem is that this approach changes the time zone
>> for all the dates in the EIT for that week. In steady state that's likely
>> to be fine, but in those two transition weeks of the year, when one batch
>> of listings has some listings before the change and some after the change,
>> some of those listings will be assigned the wrong offset.
>>
>> The correct but slightly more laborious fix, in my opinion, would be to
>> apply the offset to each individual programme listing in the batch, thus
>> possibly having programmes in the same batch with different UTC offsets,
>> which will happen during those transition weeks. I'll code it up in python
>> when I have a moment, starting from the code that Stephen posted yesterday
>> that got rid of the absurdly long lines.
> No, there will not be different offsets. Cross-eit.py is exporting
> data from the MythTV database. MythTV internally only works in UTC,
> so all the exported data will be UTC. So all that needs to be done is
> to add the string " +0000" on the end of the "start=" and "stop="
> timestamps.
>
> When mythfilldatabase reads an XMLTV file, it will use the timezone
> offsets to convert each timestamp to UTC before it puts it into the
> database. EPG collected from a TV channel (using EIT, MHEG5 or OpenTV
> which are the formats I have used) is usually in local time and
> therefore will have the correct offsets on it. When that collected
> data crosses a daylight saving change, you will see the offset values
> change (unless the EPG provider has screwed things up, which is fairly
> common in this part of the world).
>
> It is possible that the old cross-eit.py code that included using
> timedelta was from when MythTV used local times in its database. That
> caused hideous complications in the code for handling daylight saving,
> so the internal times were all changed to UTC many versions of MythTV
> ago now.
>
> I have put an updated version of cross-eit.py (v2.1) with this change
> on my web server:
>
> http://www.jsw.gen.nz/mythtv/cross-eit.py
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

Thanks will give that a try

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Mon, 6 Sept 2021 at 11:57, John <jksjdevelop@gmail.com> wrote:

> On 06/09/2021 10:46, Stephen Worthington wrote:
> > On Mon, 6 Sep 2021 07:01:01 +0100, you wrote:
> >
> >> On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
> >>
> >>> I use 65692 which does work but the number may differ for you, test
> the SD
> >>> channel you pick as source for the EIT of the HD channel.
> >>>
> >> How can I tell which one it is? I see no clue from the query below as to
> >> which of them is being used for EIT.
> >>
> >> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
> >> channel where callsign like("%Channel 4%");
> > [snip]
> >> I also tried looking at all the existing column headings in "channel"
> (with
> >> "select * from channel") but none suggested that a channel was being
> used
> >> for EIT. Obviously that info must be in some other table. Would you know
> >> which?
> > The field you want is "useonairguide".
> >
> >>> I am testing the following to cope with the clocks changing.
> >>>
> >>> I know its cringe worthy but might work, the two versions of crosseit
> have
> >>> the single line change.
> >>>
> >>> # Winter --
> >>> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
> >>> # Summer
> >>> date = dateobj + datetime.timedelta(seconds=time.altzone)
> >>>
> >>> #!/bin/bash
> >>> if date +%Z | grep -e BST; then
> >>> echo "Summer"
> >>> ./cross-eit3S.py 65692,"hd.channel4.com"
> >>> else
> >>> echo "Winter"
> >>> ./cross-eit3W.py 65692,"hd.channel4.com"
> >>> fi
> >>> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
> >>> --sourceid 1 --xmlfile output.xml
> >>>
> >> You are right that the mix of bash and python is somewhat
> cringe-worthy: it
> >> would be neater to do the change in python rather than calling two
> versions
> >> of the script.
> >>
> >> However a more serious problem is that this approach changes the time
> zone
> >> for all the dates in the EIT for that week. In steady state that's
> likely
> >> to be fine, but in those two transition weeks of the year, when one
> batch
> >> of listings has some listings before the change and some after the
> change,
> >> some of those listings will be assigned the wrong offset.
> >>
> >> The correct but slightly more laborious fix, in my opinion, would be to
> >> apply the offset to each individual programme listing in the batch, thus
> >> possibly having programmes in the same batch with different UTC offsets,
> >> which will happen during those transition weeks. I'll code it up in
> python
> >> when I have a moment, starting from the code that Stephen posted
> yesterday
> >> that got rid of the absurdly long lines.
> > No, there will not be different offsets. Cross-eit.py is exporting
> > data from the MythTV database. MythTV internally only works in UTC,
> > so all the exported data will be UTC. So all that needs to be done is
> > to add the string " +0000" on the end of the "start=" and "stop="
> > timestamps.
> >
> > When mythfilldatabase reads an XMLTV file, it will use the timezone
> > offsets to convert each timestamp to UTC before it puts it into the
> > database. EPG collected from a TV channel (using EIT, MHEG5 or OpenTV
> > which are the formats I have used) is usually in local time and
> > therefore will have the correct offsets on it. When that collected
> > data crosses a daylight saving change, you will see the offset values
> > change (unless the EPG provider has screwed things up, which is fairly
> > common in this part of the world).
> >
> > It is possible that the old cross-eit.py code that included using
> > timedelta was from when MythTV used local times in its database. That
> > caused hideous complications in the code for handling daylight saving,
> > so the internal times were all changed to UTC many versions of MythTV
> > ago now.
> >
> > I have put an updated version of cross-eit.py (v2.1) with this change
> > on my web server:
> >
> > http://www.jsw.gen.nz/mythtv/cross-eit.py
> > _______________________________________________
> > mythtv-users mailing list
> > mythtv-users@mythtv.org
> > http://lists.mythtv.org/mailman/listinfo/mythtv-users
> > http://wiki.mythtv.org/Mailing_List_etiquette
> > MythTV Forums: https://forum.mythtv.org
>
> Thanks will give that a try
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org


Just curious, do real TV's and set-top-boxes with satellite tuners show the
EPG for Channel Four HD on Freesat?

Klaas.
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
Having checked more thoroughly the new version adds an hour incorrectly,
So a program scheduled for 18:00 in the donor listing is erroneously
scheduled at 19:00.

On 06/09/2021 10:46, Stephen Worthington wrote:
> On Mon, 6 Sep 2021 07:01:01 +0100, you wrote:
>
>> On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
>>
>>> I use 65692 which does work but the number may differ for you, test the SD
>>> channel you pick as source for the EIT of the HD channel.
>>>
>> How can I tell which one it is? I see no clue from the query below as to
>> which of them is being used for EIT.
>>
>> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
>> channel where callsign like("%Channel 4%");
> [snip]
>> I also tried looking at all the existing column headings in "channel" (with
>> "select * from channel") but none suggested that a channel was being used
>> for EIT. Obviously that info must be in some other table. Would you know
>> which?
> The field you want is "useonairguide".
>
>>> I am testing the following to cope with the clocks changing.
>>>
>>> I know its cringe worthy but might work, the two versions of crosseit have
>>> the single line change.
>>>
>>> # Winter --
>>> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
>>> # Summer
>>> date = dateobj + datetime.timedelta(seconds=time.altzone)
>>>
>>> #!/bin/bash
>>> if date +%Z | grep -e BST; then
>>> echo "Summer"
>>> ./cross-eit3S.py 65692,"hd.channel4.com"
>>> else
>>> echo "Winter"
>>> ./cross-eit3W.py 65692,"hd.channel4.com"
>>> fi
>>> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
>>> --sourceid 1 --xmlfile output.xml
>>>
>> You are right that the mix of bash and python is somewhat cringe-worthy: it
>> would be neater to do the change in python rather than calling two versions
>> of the script.
>>
>> However a more serious problem is that this approach changes the time zone
>> for all the dates in the EIT for that week. In steady state that's likely
>> to be fine, but in those two transition weeks of the year, when one batch
>> of listings has some listings before the change and some after the change,
>> some of those listings will be assigned the wrong offset.
>>
>> The correct but slightly more laborious fix, in my opinion, would be to
>> apply the offset to each individual programme listing in the batch, thus
>> possibly having programmes in the same batch with different UTC offsets,
>> which will happen during those transition weeks. I'll code it up in python
>> when I have a moment, starting from the code that Stephen posted yesterday
>> that got rid of the absurdly long lines.
> No, there will not be different offsets. Cross-eit.py is exporting
> data from the MythTV database. MythTV internally only works in UTC,
> so all the exported data will be UTC. So all that needs to be done is
> to add the string " +0000" on the end of the "start=" and "stop="
> timestamps.
>
> When mythfilldatabase reads an XMLTV file, it will use the timezone
> offsets to convert each timestamp to UTC before it puts it into the
> database. EPG collected from a TV channel (using EIT, MHEG5 or OpenTV
> which are the formats I have used) is usually in local time and
> therefore will have the correct offsets on it. When that collected
> data crosses a daylight saving change, you will see the offset values
> change (unless the EPG provider has screwed things up, which is fairly
> common in this part of the world).
>
> It is possible that the old cross-eit.py code that included using
> timedelta was from when MythTV used local times in its database. That
> caused hideous complications in the code for handling daylight saving,
> so the internal times were all changed to UTC many versions of MythTV
> ago now.
>
> I have put an updated version of cross-eit.py (v2.1) with this change
> on my web server:
>
> http://www.jsw.gen.nz/mythtv/cross-eit.py
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Tue, 7 Sep 2021 23:14:15 +0100, you wrote:

>On 06/09/2021 10:46, Stephen Worthington wrote:
>> On Mon, 6 Sep 2021 07:01:01 +0100, you wrote:
>>
>>> On Sun, 5 Sept 2021 at 11:10, John <jksjdevelop@gmail.com> wrote:
>>>
>>>> I use 65692 which does work but the number may differ for you, test the SD
>>>> channel you pick as source for the EIT of the HD channel.
>>>>
>>> How can I tell which one it is? I see no clue from the query below as to
>>> which of them is being used for EIT.
>>>
>>> select serviceid,chanid,channum,sourceid,callsign,deleted,xmltvid from
>>> channel where callsign like("%Channel 4%");
>> [snip]
>>> I also tried looking at all the existing column headings in "channel" (with
>>> "select * from channel") but none suggested that a channel was being used
>>> for EIT. Obviously that info must be in some other table. Would you know
>>> which?
>> The field you want is "useonairguide".
>>
>>>> I am testing the following to cope with the clocks changing.
>>>>
>>>> I know its cringe worthy but might work, the two versions of crosseit have
>>>> the single line change.
>>>>
>>>> # Winter --
>>>> # date = dateobj # + datetime.timedelta(seconds=time.altzone)
>>>> # Summer
>>>> date = dateobj + datetime.timedelta(seconds=time.altzone)
>>>>
>>>> #!/bin/bash
>>>> if date +%Z | grep -e BST; then
>>>> echo "Summer"
>>>> ./cross-eit3S.py 65692,"hd.channel4.com"
>>>> else
>>>> echo "Winter"
>>>> ./cross-eit3W.py 65692,"hd.channel4.com"
>>>> fi
>>>> /usr/local/bin/mythfilldatabase --refresh-all --verbose xmltv --file
>>>> --sourceid 1 --xmlfile output.xml
>>>>
>>> You are right that the mix of bash and python is somewhat cringe-worthy: it
>>> would be neater to do the change in python rather than calling two versions
>>> of the script.
>>>
>>> However a more serious problem is that this approach changes the time zone
>>> for all the dates in the EIT for that week. In steady state that's likely
>>> to be fine, but in those two transition weeks of the year, when one batch
>>> of listings has some listings before the change and some after the change,
>>> some of those listings will be assigned the wrong offset.
>>>
>>> The correct but slightly more laborious fix, in my opinion, would be to
>>> apply the offset to each individual programme listing in the batch, thus
>>> possibly having programmes in the same batch with different UTC offsets,
>>> which will happen during those transition weeks. I'll code it up in python
>>> when I have a moment, starting from the code that Stephen posted yesterday
>>> that got rid of the absurdly long lines.
>>
>> No, there will not be different offsets. Cross-eit.py is exporting
>> data from the MythTV database. MythTV internally only works in UTC,
>> so all the exported data will be UTC. So all that needs to be done is
>> to add the string " +0000" on the end of the "start=" and "stop="
>> timestamps.
>>
>> When mythfilldatabase reads an XMLTV file, it will use the timezone
>> offsets to convert each timestamp to UTC before it puts it into the
>> database. EPG collected from a TV channel (using EIT, MHEG5 or OpenTV
>> which are the formats I have used) is usually in local time and
>> therefore will have the correct offsets on it. When that collected
>> data crosses a daylight saving change, you will see the offset values
>> change (unless the EPG provider has screwed things up, which is fairly
>> common in this part of the world).
>>
>> It is possible that the old cross-eit.py code that included using
>> timedelta was from when MythTV used local times in its database. That
>> caused hideous complications in the code for handling daylight saving,
>> so the internal times were all changed to UTC many versions of MythTV
>> ago now.
>>
>> I have put an updated version of cross-eit.py (v2.1) with this change
>> on my web server:
>>
>> http://www.jsw.gen.nz/mythtv/cross-eit.py

>Having checked more thoroughly the new version adds an hour incorrectly,
>So a program scheduled for 18:00 in the donor listing is erroneously
>scheduled at 19:00.

Since cross-eit.py does not change any times and works only in UTC, it
is not possible that it is cross-eit.py doing this. So if that is
what you are seeing, we will need to debug what is happening
thoroughly. The first thing to try is to dump the EPG data for both
the source and destination channels and see if there is anything
different between them. You can do that with mysqldump:

sudo mysqldump mythconverg program -X -w "chanid=<source chandid>"
>source.xml
sudo mysqldump mythconverg program -X -w "chanid=<destination
chandid>" >dest.xml
diff -u source.xml dest.xml
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
> Since cross-eit.py does not change any times and works only in UTC, it
> is not possible that it is cross-eit.py doing this. So if that is
> what you are seeing, we will need to debug what is happening
> thoroughly. The first thing to try is to dump the EPG data for both
> the source and destination channels and see if there is anything
> different between them. You can do that with mysqldump:
>
> sudo mysqldump mythconverg program -X -w "chanid=<source chandid>"
>> source.xml
> sudo mysqldump mythconverg program -X -w "chanid=<destination
> chandid>" >dest.xml
> diff -u source.xml dest.xml
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

Ok I have done some testing.

The EPG is certainly being stored in the database as UTC so the News
which is shown at 19:00 BST is stored as starting at 18:00.

The old version of cross-eit with date = dateobj +
datetime.timedelta(seconds=time.altzone) writes the correct times to the
database during BST.

JSW version with start="20210915190000 +0000" writes incorrect times to
the database.

Nemo version with start="20210915180000 +0000" writes correct times to
the database during BST.

Debug follows:-

EIT data for Channel 4 News which starts at 19:00 BST

Output from Cross-eit with? date = dateobj +
datetime.timedelta(seconds=time.altzone)

channel="hd.channel4.com" start="20210915180000"
stop="20210915190000"><title>Channel 4 News</title><sub-title>Channel
4's flagship news programme</sub-title><desc>Including the
weather.</desc><category>News</category></programme><programme
channel="hd.channel4.com" start="20210915190000"
Donor channel
MariaDB [mythconverg]> select chanid,starttime,endtime from program
where chanid=65692 and title like("Channel 4 News") and starttime
like("2021-09-15%");
+--------+---------------------+---------------------+
| chanid | starttime?????????? | endtime???????????? |
+--------+---------------------+---------------------+
|? 65692 | 2021-09-15 18:00:00 | 2021-09-15 19:00:00 |
+--------+---------------------+---------------------+
1 row in set (0.001 sec)

Client Channel
MariaDB [mythconverg]> select chanid,starttime,endtime from program
where chanid=65888 and title like("Channel 4 News") and starttime
like("2021-09-15%");
+--------+---------------------+---------------------+
| chanid | starttime?????????? | endtime???????????? |
+--------+---------------------+---------------------+
|? 65888 | 2021-09-15 18:00:00 | 2021-09-15 19:00:00 |
+--------+---------------------+---------------------+
1 row in set (0.001 sec)



The output from cross-eit-JSW
? <programme channel="hd.channel4.com" start="20210915190000 +0000"
stop="20210915200000 +0000">
??? <title>Channel 4 News</title>
??? <sub-title>Channel 4's flagship news programme</sub-title>
??? <desc>Including the weather.</desc>
??? <category>News</category>
? </programme>

MariaDB [mythconverg]> select chanid,starttime,endtime from program
where chanid=65888 and title like("Channel 4 News") and starttime
like("2021-09-15%");
+--------+---------------------+---------------------+
| chanid | starttime?????????? | endtime???????????? |
+--------+---------------------+---------------------+
|? 65888 | 2021-09-15 19:00:00 | 2021-09-15 20:00:00 |
+--------+---------------------+---------------------+
1 row in set (0.001 sec)

nemo myth797@gmail.com
Just to muddy the waters a bit, I did do some work a while ago to update
the python script to use python3 and to use the services api. I also
sorted out some of the problems with incorrect times and the like.

I've put a copy here https://pastebin.com/0tQFPBRB
The output from cross-eit-Nemo
<programme channel="hd.channel4.com" start="20210915180000 +0000"
stop="20210915190000 +0000"><title>Channel 4
News</title><sub-title>Channel 4's flagship news
programme</sub-title><desc>Including the
weather.</desc><category>News</category><date
/><episode-num>0</episode-num></programme>

MariaDB [mythconverg]> select chanid,starttime,endtime from program
where chanid=65888 and title like("Channel 4 News") and starttime
like("2021-09-15%");
+--------+---------------------+---------------------+
| chanid | starttime?????????? | endtime???????????? |
+--------+---------------------+---------------------+
|? 65888 | 2021-09-15 18:00:00 | 2021-09-15 19:00:00 |
+--------+---------------------+---------------------+
1 row in set (0.001 sec)

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Thu, 9 Sept 2021 at 18:29, John <jksjdevelop@gmail.com> wrote:

>
> Ok I have done some testing.
>
> The EPG is certainly being stored in the database as UTC so the News
> which is shown at 19:00 BST is stored as starting at 18:00.
>
> The old version of cross-eit with date = dateobj +
> datetime.timedelta(seconds=time.altzone) writes the correct times to the
> database during BST.
>
> JSW version with start="20210915190000 +0000" writes incorrect times to
> the database.
>
> Nemo version with start="20210915180000 +0000" writes correct times to
> the database during BST.
>


I am still getting off-by-one-hour failures, though only for SOME channel 4
HD recordings, not all, and this is all very confusing. I have also been
doing some debugging but I can't figure out where this extra hour
difference comes from. Here is what I've got to so far.

Ground truth: Karate Kid II was broadcast on Channel 4 HD today
(2021-09-12) starting at 1350 UTC+1.
Indeed the channel 4 website says, at
https://www.channel4.com/programmes/the-karate-kid-part-ii , "Sun 12 Sep,
1.50pm"

It appeared in my Myth schedule (using the old cross-eit at
https://pastebin.com/d51VUASq ) as starting at 1450, hence still with the
error. The recording scheduled from the myth GUI would have missed the
first hour.

The above "old" cross-eit produced

<programme channel="hd.channel4.com" start="20210912135000" stop=
"20210912160000"><title>The Karate Kid Part II</title>

(start time correct IFF you assume UTC+1; incorrect in UTC+0)


The JSW 2.0 cross-eit produced

<programme channel="hd.channel4.com" start="20210912135000" stop=
"20210912160000">

<title>The Karate Kid Part II</title>

(start time correct IFF you assume UTC+1; incorrect in UTC+0)


The JSW 2.1 cross-eit produced


<programme channel="hd.channel4.com" start="20210912135000 +0000" stop=
"20210912160000 +0000">

<title>The Karate Kid Part II</title>

(start time in UTC+0 that's definitely incorrect: if anything it ought to
be 1250)



Last night, a cron job ran these two commands:


/home/tv/bin/cross-eit.py -o /tmp/export.xml 19211,"hd.channel4.com"

/usr/bin/mythfilldatabase --refresh-all --verbose xmltv --file --sourceid 1
--xmlfile /tmp/export.xml


and the first of them produced this file (still untouched)...


-rw-rw-r-- 1 tv tv 216039 Sep 12 04:30 /tmp/export.xml

and the following database content:


mysql> select p.chanid, c.name, p.starttime, p.endtime, p.title from
program as p, channel as c where p.chanid=c.chanid and p.title
like("%karate kid part ii");

+--------+--------------+---------------------+---------------------+------------------------+

| chanid | name | starttime | endtime | title
|

+--------+--------------+---------------------+---------------------+------------------------+

| 18320 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19211 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19212 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19213 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19214 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19216 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 18311 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18312 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18314 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18316 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 31200 | Channel 4 HD | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

+--------+--------------+---------------------+---------------------+------------------------+

11 rows in set (0.29 sec)


Here the start time on channels 19211 and 31200 is 1250 rather than 1350,
but if it's true that the database always contains UTC+0 then that's fine,
that would be equivalent to 1350 UTC+1, hence still the correct time. Yet
the GUI showed a start time of 1450. Why??



Further experimentation: I then imported the xml produced by JSW 2.1 (the
one with a start time of 1350 UTC+0, which is INCORRECT) with the following
command:


/usr/bin/mythfilldatabase --refresh-all --verbose xmltv --file --sourceid
1 --xmlfile export-jsw21.xml


and, after that, the same query resulted in


mysql> select p.chanid, c.name, p.starttime, p.endtime, p.title from
program as p, channel as c where p.chanid=c.chanid and p.title
like("%karate kid part ii");

+--------+--------------+---------------------+---------------------+------------------------+

| chanid | name | starttime | endtime | title
|

+--------+--------------+---------------------+---------------------+------------------------+

| 31200 | Channel 4 HD | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18320 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19211 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19212 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19213 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19214 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 19216 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
Karate Kid Part II |

| 18311 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18312 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18314 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

| 18316 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
Karate Kid Part II |

+--------+--------------+---------------------+---------------------+------------------------+

11 rows in set (0.29 sec)


You will note that now 19211 stays at 1250 (correct in UTC+0) but 31200 has
gone to 1350 (incorrect in UTC+0, although unsurprising as that's what the
xml said). Now what does the GUI say? The program guide still says the
start time is 1450 local time (incorrect).


My unanswered questions:


1) why, if the imported XML says 1350 (not qualified with a time zone),
mythfilldatabase translates that to 1250 UTC+0?

(Educated guess: because myth assumes that an unqualified date is in local
time, which would make a lot of sense. But then, if myth is smart enough to
translate from local time to UTC, why do we ever have off-by-one errors? By
this logic, these should never occur.)


2) why does cross-eit JSW 2.1 output 1350 UTC+0 into the XML if the EIT
stored in the database (already in UTC+0) says 1250? Shouldn't cross-eit
just repeat 1250 and just add UTC+0?


3) what's the proper way to fix all this? I can add and subtract 1 hour as
well as anyone else, but by now it's totally unclear under which
circumstances it should be done.


4) is there a way to see (and possibly even log) what the on-air EIT
originally said, before being imported in the database as UTC+0? Did the
EIT mention a time zone or was it in local time?
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On Sun, 12 Sep 2021 22:16:46 +0100, you wrote:

>On Thu, 9 Sept 2021 at 18:29, John <jksjdevelop@gmail.com> wrote:
>
>>
>> Ok I have done some testing.
>>
>> The EPG is certainly being stored in the database as UTC so the News
>> which is shown at 19:00 BST is stored as starting at 18:00.
>>
>> The old version of cross-eit with date = dateobj +
>> datetime.timedelta(seconds=time.altzone) writes the correct times to the
>> database during BST.
>>
>> JSW version with start="20210915190000 +0000" writes incorrect times to
>> the database.
>>
>> Nemo version with start="20210915180000 +0000" writes correct times to
>> the database during BST.
>>
>
>
>I am still getting off-by-one-hour failures, though only for SOME channel 4
>HD recordings, not all, and this is all very confusing. I have also been
>doing some debugging but I can't figure out where this extra hour
>difference comes from. Here is what I've got to so far.
>
>Ground truth: Karate Kid II was broadcast on Channel 4 HD today
>(2021-09-12) starting at 1350 UTC+1.
>Indeed the channel 4 website says, at
>https://www.channel4.com/programmes/the-karate-kid-part-ii , "Sun 12 Sep,
>1.50pm"
>
>It appeared in my Myth schedule (using the old cross-eit at
>https://pastebin.com/d51VUASq ) as starting at 1450, hence still with the
>error. The recording scheduled from the myth GUI would have missed the
>first hour.
>
>The above "old" cross-eit produced
>
><programme channel="hd.channel4.com" start="20210912135000" stop=
>"20210912160000"><title>The Karate Kid Part II</title>
>
>(start time correct IFF you assume UTC+1; incorrect in UTC+0)
>
>
>The JSW 2.0 cross-eit produced
>
> <programme channel="hd.channel4.com" start="20210912135000" stop=
>"20210912160000">
>
> <title>The Karate Kid Part II</title>
>
>(start time correct IFF you assume UTC+1; incorrect in UTC+0)
>
>
>The JSW 2.1 cross-eit produced
>
>
> <programme channel="hd.channel4.com" start="20210912135000 +0000" stop=
>"20210912160000 +0000">
>
> <title>The Karate Kid Part II</title>
>
>(start time in UTC+0 that's definitely incorrect: if anything it ought to
>be 1250)
>
>
>
>Last night, a cron job ran these two commands:
>
>
>/home/tv/bin/cross-eit.py -o /tmp/export.xml 19211,"hd.channel4.com"
>
>/usr/bin/mythfilldatabase --refresh-all --verbose xmltv --file --sourceid 1
>--xmlfile /tmp/export.xml
>
>
>and the first of them produced this file (still untouched)...
>
>
>-rw-rw-r-- 1 tv tv 216039 Sep 12 04:30 /tmp/export.xml
>
>and the following database content:
>
>
>mysql> select p.chanid, c.name, p.starttime, p.endtime, p.title from
>program as p, channel as c where p.chanid=c.chanid and p.title
>like("%karate kid part ii");
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>| chanid | name | starttime | endtime | title
> |
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>| 18320 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19211 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19212 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19213 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19214 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19216 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 18311 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18312 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18314 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18316 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 31200 | Channel 4 HD | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>11 rows in set (0.29 sec)
>
>
>Here the start time on channels 19211 and 31200 is 1250 rather than 1350,
>but if it's true that the database always contains UTC+0 then that's fine,
>that would be equivalent to 1350 UTC+1, hence still the correct time. Yet
>the GUI showed a start time of 1450. Why??
>
>
>
>Further experimentation: I then imported the xml produced by JSW 2.1 (the
>one with a start time of 1350 UTC+0, which is INCORRECT) with the following
>command:
>
>
> /usr/bin/mythfilldatabase --refresh-all --verbose xmltv --file --sourceid
>1 --xmlfile export-jsw21.xml
>
>
>and, after that, the same query resulted in
>
>
>mysql> select p.chanid, c.name, p.starttime, p.endtime, p.title from
>program as p, channel as c where p.chanid=c.chanid and p.title
>like("%karate kid part ii");
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>| chanid | name | starttime | endtime | title
> |
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>| 31200 | Channel 4 HD | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18320 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19211 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19212 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19213 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19214 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 19216 | Channel 4 | 2021-09-12 12:50:00 | 2021-09-12 15:00:00 | The
>Karate Kid Part II |
>
>| 18311 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18312 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18314 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>| 18316 | Channel 4+1 | 2021-09-12 13:50:00 | 2021-09-12 16:00:00 | The
>Karate Kid Part II |
>
>+--------+--------------+---------------------+---------------------+------------------------+
>
>11 rows in set (0.29 sec)
>
>
>You will note that now 19211 stays at 1250 (correct in UTC+0) but 31200 has
>gone to 1350 (incorrect in UTC+0, although unsurprising as that's what the
>xml said). Now what does the GUI say? The program guide still says the
>start time is 1450 local time (incorrect).
>
>
>My unanswered questions:
>
>
>1) why, if the imported XML says 1350 (not qualified with a time zone),
>mythfilldatabase translates that to 1250 UTC+0?
>
>(Educated guess: because myth assumes that an unqualified date is in local
>time, which would make a lot of sense. But then, if myth is smart enough to
>translate from local time to UTC, why do we ever have off-by-one errors? By
>this logic, these should never occur.)
>
>
>2) why does cross-eit JSW 2.1 output 1350 UTC+0 into the XML if the EIT
>stored in the database (already in UTC+0) says 1250? Shouldn't cross-eit
>just repeat 1250 and just add UTC+0?

Yes, that is what it does. It just takes the UTC timestamp from the
database and adds the string +0000 to say that the timestamp is in
UTC. If there is a problem with the data, the problem must be already
present in the data in the program table.

>3) what's the proper way to fix all this? I can add and subtract 1 hour as
>well as anyone else, but by now it's totally unclear under which
>circumstances it should be done.

You need to check the data in the database before using cross-eit.py
to export it. It looks to me that the data in the database is the
problem, not cross-eit.py.

>4) is there a way to see (and possibly even log) what the on-air EIT
>originally said, before being imported in the database as UTC+0? Did the
>EIT mention a time zone or was it in local time?

If you have a Windows machine with access to a tuner (eg network
tuner), you can fetch the EIT data outside MythTV using EPG Collector:

https://sourceforge.net/projects/epgcollector/

You will need to ensure that MythTV is not trying to use the tuner
when EPG Collector is using it. You need to tell MythTV to release
the tuner between recordings and to not use it for EIT.

To get EIT data from tuners in Linux, dvbsnoop can do that. I use
epgsnoop to run dvbsnoop for doing that:

https://github.com/hadleyrich/epgsnoop

I have never used epgsnoop with my DVB-T(2) tuners, as in New Zealand
our EPG on DVB-T is in MHEG5 format. But it should work. You do have
to set up a channels.conf file with all the channel data. You can
create that from the MythTV channel table with some SQL:

sudo mysql mythconverg
select chanid,xmltvid,icon,'',channum from channel where sourceid=1
order by chanid into outfile '/tmp/channels.conf' fields terminated by
'|' lines terminated by '\n';
quit

From my database, that creates this in channels.conf:

1001|tv1.freeviewnz.tv|tvnz_tv_1.png||1
1002|tv2.freeviewnz.tv|tvnz_tv_2.png||2
1003|tv3.freeviewnz.tv|THREE_1280x1280.png||3
1004|bravo.freeviewnz.tv|bravo.jpg||4
1005|maori-tv.freeviewnz.tv|maori_tv.jpg||5
1006|tv1-plus1.freeviewnz.tv|tvnz1p1.jpg||6
1007|tv2-plus1.freeviewnz.tv|tvnz2p1.jpg||7
1008|tv3-plus1.freeviewnz.tv|threep1.jpg||8
1009|bravo-plus1.freeviewnz.tv|bravop1.jpg||9
1010|prime.freeviewnz.tv|prime.jpg||10
1011|theedgetv.freeviewnz.tv|theedgetv.png||11
1012|choice.freeviewnz.tv|ChoiceTV_logo.png||12
1013|duke.freeviewnz.tv|tvnz_tv_duke.png||13
1014|breezetv.freeviewnz.tv|breezetv.png||14
1015|te-reo.freeviewnz.tv|tereo.jpg||15
1016|aljazeera.freeviewnz.tv|al_jazeera.jpg||16
1017|hgtv.freeviewnz.tv|hgtv.jpg||17
1018|duke-plus1.freeviewnz.tv|tvnz_tv_duke_plus_1.png||18
1020|tvsn-shopping-channel.freeviewnz.tv|TVSN.jpg||20
1021|shine.freeviewnz.tv|shinetv.jpg||21
1022|parliament.freeviewnz.tv|parliament.jpg||25
1024|te-reo.freeviewnz.tv|tereo.jpg||24
1026|firstlight.freeviewnz.tv|Firstlight.jpg||26
1027|hope-channel.freeviewnz.tv|hope_channel.jpg||27
1028|chinese-tv8.freeviewnz.tv|ctv8.gif||28
1029|tv29.freeviewnz.tv|tv29.jpg||29
1033|c33.freeviewnz.tv|C33.jpg||33
1036|apna.freeviewnz.tv|Apna.jpg||36
1050|rnz-national.freeviewnz.tv|radio_nz_national.jpg||50
1051|rnz-concert.freeviewnz.tv|radio_nz_concert.jpg||51
1071|basefm.freeviewnz.tv|basefm.jpg||71
1107|tvnz7.freeviewnz.tv|tvnz7.jpg||107
1114|tvnz-games-extra.freeviewnz.tv|tvnz_tv_games_extra.png||114
1200|kordiatv.freeviewnz.tv|KordiaTV.png||200

You need to then copy the headers from the channel.conf examples that
come with epgsnoop - add them above the data created by the SQL query.

[Update]

I have now tried epgsnoop with DVB-T and it does not work - it is set
up for DVB-S(2) only. Manual use of dvbtune and dvbsnoop would work,
but it would be a pain and the output format would not be XMLTV. And
epgsnoop is Python 2. So I am working on a Python 3 version of
epgsnoop that will support DVB-T as well. I may not be able to test
it properly here in New Zealand as I am not sure if we now have EIT
EPG as well as MHEG5.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Fwd: Fwd: Re: channel4 hd has no eit - cross-eit [ In reply to ]
On 13/09/2021 10:58, Stephen Worthington wrote:
> I am still getting off-by-one-hour failures, though only for SOME channel 4
> HD recordings, not all, and this is all very confusing. I have also been
> doing some debugging but I can't figure out where this extra hour
> difference comes from. Here is what I've got to so far.

Note that one possible source of confusion is that 'Now and Next' data
is still being broadcast for Chanel 4HD so even though there is no EPG
data, the database is being updated for the current time and one hour
ahead. So make sure that the test case is well away from time now.

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org