Mailing List Archive

Has anyone tested channel scanning in a us-cable Clear QAM environment?
I ask because I can't get mythtv-setup to find any channels. I first
tried a scan before running mythfilldatabase as mentioned in this email
thread on mythtv-users:
http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html

I just tried another channel scan after running mythfilldatabase.
mythfilldatabase added lots of channels and listings to the database,
but the channel data looked incomplete. So I did a full channel scan,
which said it was scanning for ATSC channels, not QAM, and found no
channels. I can't find any channel scan settings that distinguish
between ATSC and QAM.

Not surprisingly, the channel data still looks wrong. For example, all
of the channel records have a NULL mplexid and the dtv_multiplex table
is empty. Has tuning a clear qam channel changed so much that channel
records don't need to point to valid dtv_multiplex records any more?

FWIW, both "General -> Locale Settings -> Channel frequency table" and
"Video Sources -> MyVideoSource -> Channel frequency table" are set to
us-cable and the listings grabber is "Multinational (Schedules Direct
JSON web services w..." and the listings data looks correct.


_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com> wrote:

> I ask because I can't get mythtv-setup to find any channels. I first
> tried a scan before running mythfilldatabase as mentioned in this email
> thread on mythtv-users:
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
>
> I just tried another channel scan after running mythfilldatabase.
> mythfilldatabase added lots of channels and listings to the database,
> but the channel data looked incomplete. So I did a full channel scan,
> which said it was scanning for ATSC channels, not QAM, and found no
> channels. I can't find any channel scan settings that distinguish
> between ATSC and QAM.
>
> Not surprisingly, the channel data still looks wrong. For example, all
> of the channel records have a NULL mplexid and the dtv_multiplex table
> is empty. Has tuning a clear qam channel changed so much that channel
> records don't need to point to valid dtv_multiplex records any more?
>
> FWIW, both "General -> Locale Settings -> Channel frequency table" and
> "Video Sources -> MyVideoSource -> Channel frequency table" are set to
> us-cable and the listings grabber is "Multinational (Schedules Direct
> JSON web services w..." and the listings data looks correct.
>
>
> It is interesting to know which version of MythTV you are running. There
are lots of changes in the latest master / v31 compared with previous
versions.
You might have missed the Scan Type/ Full Scan settings, see the end of
this thread:
https://forum.mythtv.org/viewtopic.php?f=36&t=3567
Also described here:
https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On February 7, 2020 11:28:13 AM EST, Derek Atkins <derek@ihtfp.com> wrote:
>faginbagin <mythtv@hbuus.com> writes:
>
>> "Has anyone tested channel scanning in a us-cable Clear QAM
>> environment?"
>
>Does such an animal still actually exist? I haven't been able to
>subscribe to a Clear QAM cable plant in a decade (possibly more).
>
>-derek

Yes, it does. My cable provider. WideOpenWest (aka WOW, wowway.com), broadcasts the local OTA channels in the clear and in HD. It also broadcasts basic cable channels in SD in the clear. But premium and HD versions of basic cable are encrypted. I'm in their metro Detroit market and I'm pretty sure it's the same in other WOW markets, but I could be wrong. Here's a discussion from a year ago on the subject:

https://www.dslreports.com/forum/r32258900-Local-channels-without-a-box-Clear-QAM-in-Cleveland-area

I'm OK with SD versions of basic cable, so I never felt the need to replace my tuners with ones that need a cable card. And I've got too many 70' trees surrounding my property to ever hope to get reliable OTA reception.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com>> wrote:
>
> I ask because I can't get mythtv-setup to find any channels. I first
> tried a scan before running mythfilldatabase as mentioned in this email
> thread on mythtv-users:
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
>
> I just tried another channel scan after running mythfilldatabase.
> mythfilldatabase added lots of channels and listings to the database,
> but the channel data looked incomplete. So I did a full channel scan,
> which said it was scanning for ATSC channels, not QAM, and found no
> channels. I can't find any channel scan settings that distinguish
> between ATSC and QAM.
>
> Not surprisingly, the channel data still looks wrong. For example, all
> of the channel records have a NULL mplexid and the dtv_multiplex table
> is empty. Has tuning a clear qam channel changed so much that channel
> records don't need to point to valid dtv_multiplex records any more?
>
> FWIW, both "General -> Locale Settings -> Channel frequency table" and
> "Video Sources -> MyVideoSource -> Channel frequency table" are set to
> us-cable and the listings grabber is "Multinational (Schedules Direct
> JSON web services w..." and the listings data looks correct.
>
>
> It is interesting to know which version of MythTV you are running. There are lots of changes in the latest master / v31 compared with previous versions.
> You might have missed the Scan Type/ Full Scan settings, see the end of this thread:
> https://forum.mythtv.org/viewtopic.php?f=36&t=3567  
> Also described here:
> https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2  
>
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>

Hi Klaas,

My apologies for not mentioning the version, it's v31 from the ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro is lubuntu 19.10 with the lxqt desktop.

Both the forum thread and images got me on the right track. Instead of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256 channels. It's still in progress. I'll report back once I've done further testing.

But I must say the new settings UI is quite confusing and a departure from how other parts of mythtv's UI work. It hadn't occurred to me that right arrow would do something different than OK. Also, that left arrow behaves differently than ESC.

Another point of confusion: after navigating to the scan type page and changing the frequency table and modulation to cable and qam-256, followed by pressing ESC and being asked if I want to save my changes, and saying yes, why am I kicked out of not only the scan type page, but also out of the channel scan page? And when I go back into the scan type page, why are the frequency table and modulation settings back to the original broadcast and 8-vsb settings? Maybe something can be done to explain these points in the help text and the wiki?

And, if I've specified us-cable as the channel frequency table in the channel scan page and also in the Global -> Locale page, why does the frequency table in the scan type page always revert to Broadcast, not to mention 8-vsb modulation? Seems to me there's room for improvement in setting more reasonable defaults based on the us-cable setting as well as saving the settings the user specifies?

Food for thought?
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
Hi faginbagin,

On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv@hbuus.com> wrote:

> On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com <mailto:
> mythtv@hbuus.com>> wrote:
> >
> > I ask because I can't get mythtv-setup to find any channels. I first
> > tried a scan before running mythfilldatabase as mentioned in this
> email
> > thread on mythtv-users:
> >
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> >
> > I just tried another channel scan after running mythfilldatabase.
> > mythfilldatabase added lots of channels and listings to the database,
> > but the channel data looked incomplete. So I did a full channel scan,
> > which said it was scanning for ATSC channels, not QAM, and found no
> > channels. I can't find any channel scan settings that distinguish
> > between ATSC and QAM.
> >
> > Not surprisingly, the channel data still looks wrong. For example,
> all
> > of the channel records have a NULL mplexid and the dtv_multiplex
> table
> > is empty. Has tuning a clear qam channel changed so much that channel
> > records don't need to point to valid dtv_multiplex records any more?
> >
> > FWIW, both "General -> Locale Settings -> Channel frequency table"
> and
> > "Video Sources -> MyVideoSource -> Channel frequency table" are set
> to
> > us-cable and the listings grabber is "Multinational (Schedules Direct
> > JSON web services w..." and the listings data looks correct.
> >
> >
> > It is interesting to know which version of MythTV you are running. There
> are lots of changes in the latest master / v31 compared with previous
> versions.
> > You might have missed the Scan Type/ Full Scan settings, see the end of
> this thread:
> > https://forum.mythtv.org/viewtopic.php?f=36&t=3567
> > Also described here:
> > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2
> >
> >
> >
> > _______________________________________________
> > mythtv-dev mailing list
> > mythtv-dev@mythtv.org
> > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > http://wiki.mythtv.org/Mailing_List_etiquette
> > MythTV Forums: https://forum.mythtv.org
> >
>
> Hi Klaas,
>
> My apologies for not mentioning the version, it's v31 from the ubuntu PPA,
> 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro is lubuntu
> 19.10 with the lxqt desktop.
>
> Both the forum thread and images got me on the right track. Instead of
> scanning 51 ATSC channels, I'm now scanning 130+ QAM-256 channels. It's
> still in progress. I'll report back once I've done further testing.
>
> But I must say the new settings UI is quite confusing and a departure from
> how other parts of mythtv's UI work. It hadn't occurred to me that right
> arrow would do something different than OK. Also, that left arrow behaves
> differently than ESC.
>
> Another point of confusion: after navigating to the scan type page and
> changing the frequency table and modulation to cable and qam-256, followed
> by pressing ESC and being asked if I want to save my changes, and saying
> yes, why am I kicked out of not only the scan type page, but also out of
> the channel scan page? And when I go back into the scan type page, why are
> the frequency table and modulation settings back to the original broadcast
> and 8-vsb settings? Maybe something can be done to explain these points in
> the help text and the wiki?
>
> And, if I've specified us-cable as the channel frequency table in the
> channel scan page and also in the Global -> Locale page, why does the
> frequency table in the scan type page always revert to Broadcast, not to
> mention 8-vsb modulation? Seems to me there's room for improvement in
> setting more reasonable defaults based on the us-cable setting as well as
> saving the settings the user specifies?
>
> Food for thought?
>

I am glad that the scanning does work for you now.
About your painful experiences with the GUI, you have made a good summary
of what can be improved. It is largely in line with what I have in mind.
For the time being I have documented the way it works in the WIki in
https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people to
actually read all of this, But it is there for when desperation sets in....
Interesting is also that most of the changes in v31 are "under the hood",
the GUI is not really changed since v30 except for a few new options.
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On 2/8/2020 3:52 AM, Klaas de Waal wrote:
> Hi faginbagin,
>
> On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com>> wrote:
>
> On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>> wrote:
> >
> >     I ask because I can't get mythtv-setup to find any channels. I first
> >     tried a scan before running mythfilldatabase as mentioned in this email
> >     thread on mythtv-users:
> >     http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> >
> >     I just tried another channel scan after running mythfilldatabase.
> >     mythfilldatabase added lots of channels and listings to the database,
> >     but the channel data looked incomplete. So I did a full channel scan,
> >     which said it was scanning for ATSC channels, not QAM, and found no
> >     channels. I can't find any channel scan settings that distinguish
> >     between ATSC and QAM.
> >
> >     Not surprisingly, the channel data still looks wrong. For example, all
> >     of the channel records have a NULL mplexid and the dtv_multiplex table
> >     is empty. Has tuning a clear qam channel changed so much that channel
> >     records don't need to point to valid dtv_multiplex records any more?
> >
> >     FWIW, both "General -> Locale Settings -> Channel frequency table" and
> >     "Video Sources -> MyVideoSource -> Channel frequency table" are set to
> >     us-cable and the listings grabber is "Multinational (Schedules Direct
> >     JSON web services w..." and the listings data looks correct.
> >
> >
> > It is interesting to know which version of MythTV you are running. There are lots of changes in the latest master / v31 compared with previous versions.
> > You might have missed the Scan Type/ Full Scan settings, see the end of this thread:
> > https://forum.mythtv.org/viewtopic.php?f=36&t=3567  
> > Also described here:
> > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2  
> >
> >
> >
> > _______________________________________________
> > mythtv-dev mailing list
> > mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org>
> > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > http://wiki.mythtv.org/Mailing_List_etiquette
> > MythTV Forums: https://forum.mythtv.org
> >
>
> Hi Klaas,
>
> My apologies for not mentioning the version, it's v31 from the ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro is lubuntu 19.10 with the lxqt desktop.
>
> Both the forum thread and images got me on the right track. Instead of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256 channels. It's still in progress. I'll report back once I've done further testing.
>
> But I must say the new settings UI is quite confusing and a departure from how other parts of mythtv's UI work. It hadn't occurred to me that right arrow would do something different than OK. Also, that left arrow behaves differently than ESC.
>
> Another point of confusion: after navigating to the scan type page and changing the frequency table and modulation to cable and qam-256, followed by pressing ESC and being asked if I want to save my changes, and saying yes, why am I kicked out of not only the scan type page, but also out of the channel scan page? And when I go back into the scan type page, why are the frequency table and modulation settings back to the original broadcast and 8-vsb settings? Maybe something can be done to explain these points in the help text and the wiki?
>
> And, if I've specified us-cable as the channel frequency table in the channel scan page and also in the Global -> Locale page, why does the frequency table in the scan type page always revert to Broadcast, not to mention 8-vsb modulation? Seems to me there's room for improvement in setting more reasonable defaults based on the us-cable setting as well as saving the settings the user specifies?
>
> Food for thought?
>
>
> I am glad that the scanning does work for you now. 
> About your painful experiences with the GUI, you have made a good summary of what can be improved. It is largely in line with what I have in mind. For the time being I have documented the way it works in the WIki in  https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people to actually read all of this, But it is there for when desperation sets in....
> Interesting is also that most of the changes in v31 are "under the hood", the GUI is not really changed since v30 except for a few new options.

Hi Klaas,

I haven't used mythtv-setup to do channel scans in years. My cable company has only made incremental changes over the years, most of which haven't been important to me. When they do make a change I care about, I've used hdhomerun_config to scan for changes, then crafted sql inserts and updates to the channel and dtv_multiplex tables. I tend to upgrade mythtv with every other release, e.g. 0.27 to 29 and now to 31. And I didn't need to spend much time in mythtv-setup to get 29 configured for my needs.

Now I'm getting ready to upgrade to 31 and find it necessary to re-learn how to setup a video source because of the elimination of the DataDirect video source. So, I figured it was also time to see how channel scanning works these days. I had hoped, since the channel table was populated by the new Schedules Direct video source, that maybe the channel scan would be able to update those channel records. But it didn't happen, at least not in my case. Instead, it just added all new channel records for those found in the scan.

In my case, the channel scan only found 68 channels when there are actually just over 100 channels. Maybe increasing timeouts would help. But if I've still got to populate the xmltvid columns, then I think I'm better off starting with my production system's channel and dtv_multiplex tables and using the switch_to_json.py script Bill Meek sent me to automate conversion from the old to the new xmltvids.

Many Thanks for your help in getting channel scanning working.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
Hi faginbagin,

On Mon, 10 Feb 2020 at 03:15, faginbagin <mythtv@hbuus.com> wrote:

> On 2/8/2020 3:52 AM, Klaas de Waal wrote:
> > Hi faginbagin,
> >
> > On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv@hbuus.com <mailto:
> mythtv@hbuus.com>> wrote:
> >
> > On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> > > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com <mailto:
> mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>>
> wrote:
> > >
> > > I ask because I can't get mythtv-setup to find any channels. I
> first
> > > tried a scan before running mythfilldatabase as mentioned in
> this email
> > > thread on mythtv-users:
> > >
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> > >
> > > I just tried another channel scan after running
> mythfilldatabase.
> > > mythfilldatabase added lots of channels and listings to the
> database,
> > > but the channel data looked incomplete. So I did a full
> channel scan,
> > > which said it was scanning for ATSC channels, not QAM, and
> found no
> > > channels. I can't find any channel scan settings that
> distinguish
> > > between ATSC and QAM.
> > >
> > > Not surprisingly, the channel data still looks wrong. For
> example, all
> > > of the channel records have a NULL mplexid and the
> dtv_multiplex table
> > > is empty. Has tuning a clear qam channel changed so much that
> channel
> > > records don't need to point to valid dtv_multiplex records any
> more?
> > >
> > > FWIW, both "General -> Locale Settings -> Channel frequency
> table" and
> > > "Video Sources -> MyVideoSource -> Channel frequency table"
> are set to
> > > us-cable and the listings grabber is "Multinational (Schedules
> Direct
> > > JSON web services w..." and the listings data looks correct.
> > >
> > >
> > > It is interesting to know which version of MythTV you are running.
> There are lots of changes in the latest master / v31 compared with previous
> versions.
> > > You might have missed the Scan Type/ Full Scan settings, see the
> end of this thread:
> > > https://forum.mythtv.org/viewtopic.php?f=36&t=3567
> > > Also described here:
> > > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2
> > >
> > >
> > >
> > > _______________________________________________
> > > mythtv-dev mailing list
> > > mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org>
> > > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > > http://wiki.mythtv.org/Mailing_List_etiquette
> > > MythTV Forums: https://forum.mythtv.org
> > >
> >
> > Hi Klaas,
> >
> > My apologies for not mentioning the version, it's v31 from the
> ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro
> is lubuntu 19.10 with the lxqt desktop.
> >
> > Both the forum thread and images got me on the right track. Instead
> of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256 channels. It's
> still in progress. I'll report back once I've done further testing.
> >
> > But I must say the new settings UI is quite confusing and a
> departure from how other parts of mythtv's UI work. It hadn't occurred to
> me that right arrow would do something different than OK. Also, that left
> arrow behaves differently than ESC.
> >
> > Another point of confusion: after navigating to the scan type page
> and changing the frequency table and modulation to cable and qam-256,
> followed by pressing ESC and being asked if I want to save my changes, and
> saying yes, why am I kicked out of not only the scan type page, but also
> out of the channel scan page? And when I go back into the scan type page,
> why are the frequency table and modulation settings back to the original
> broadcast and 8-vsb settings? Maybe something can be done to explain these
> points in the help text and the wiki?
> >
> > And, if I've specified us-cable as the channel frequency table in
> the channel scan page and also in the Global -> Locale page, why does the
> frequency table in the scan type page always revert to Broadcast, not to
> mention 8-vsb modulation? Seems to me there's room for improvement in
> setting more reasonable defaults based on the us-cable setting as well as
> saving the settings the user specifies?
> >
> > Food for thought?
> >
> >
> > I am glad that the scanning does work for you now.
> > About your painful experiences with the GUI, you have made a good
> summary of what can be improved. It is largely in line with what I have in
> mind. For the time being I have documented the way it works in the WIki in
> https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people
> to actually read all of this, But it is there for when desperation sets
> in....
> > Interesting is also that most of the changes in v31 are "under the
> hood", the GUI is not really changed since v30 except for a few new options.
>
> Hi Klaas,
>
> I haven't used mythtv-setup to do channel scans in years. My cable company
> has only made incremental changes over the years, most of which haven't
> been important to me. When they do make a change I care about, I've used
> hdhomerun_config to scan for changes, then crafted sql inserts and updates
> to the channel and dtv_multiplex tables. I tend to upgrade mythtv with
> every other release, e.g. 0.27 to 29 and now to 31. And I didn't need to
> spend much time in mythtv-setup to get 29 configured for my needs.
>
> Now I'm getting ready to upgrade to 31 and find it necessary to re-learn
> how to setup a video source because of the elimination of the DataDirect
> video source. So, I figured it was also time to see how channel scanning
> works these days. I had hoped, since the channel table was populated by the
> new Schedules Direct video source, that maybe the channel scan would be
> able to update those channel records. But it didn't happen, at least not in
> my case. Instead, it just added all new channel records for those found in
> the scan.
>
> In my case, the channel scan only found 68 channels when there are
> actually just over 100 channels. Maybe increasing timeouts would help. But
> if I've still got to populate the xmltvid columns, then I think I'm better
> off starting with my production system's channel and dtv_multiplex tables
> and using the switch_to_json.py script Bill Meek sent me to automate
> conversion from the old to the new xmltvids.
>
> Many Thanks for your help in getting channel scanning working.
> _______________________________________________
>

Thanks for testing on USA cable. I appreciate getting feedback on this
since I can test only on DVB sources. I have made changes over time that
also can influence the ATSC behaviour but there has no serious regression
testing been done on this.

About finding only 68 channels when there are actually just over 100
channels.
Could you try with larger timeouts to see if that makes a difference? In
v31 the default timeout values have increased from 1000/3000 to 3000/6000
but maybe even bigger timeouts are needed. Note that this new default is
only for new capture cards, the timeout values of existing capture card
entries are not changed.

It could also be that there are frequencies used that are not being tested
with a channel scan. The way it works is that a predefined list of
frequencies is scanned but maybe that does not cover all frequencies.
To figure this out, could you provide me with a list of channels as found
by hdhomerun_config and also the output of a mythtv-setup channel scan with
the "-v chanscan" option? E.g. "mythtv-setup -v chanscan 2>&1 | tee
ms-20200210-2033.log".

About creating new channels vs. updating existing channels.
The v31 code does try to update existing channels more than before so that
non-scanned channel information like xmltvid is preserved across scans.
If I understand you correct then you have tested with a channel list
created by/from Schedules Direct and then do a mythtv-setup channel scan.
The expected/wanted behaviour is that the channels in the database are
updated, but the actual behaviour is that mythtv-setup creates new channels.
Correct?
To figure if the expected behaviour can be realized I need a dump of the
tables channel and dtv_multiplex as created by/from SchedulesDirect and
also as created by mythtv-setup.

N.B. the default Channel frequency table for ATSC "Full Scan" is now
obtained from the Video Source table value or, if that is "default", from
the General / Locale settings table value, in master and in fixes/31. Of
course it might take a few days before this is in a repository near you.

Thanks,
Klaas.
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On 2/10/2020 2:53 PM, Klaas de Waal wrote:
> Hi faginbagin,
>
> On Mon, 10 Feb 2020 at 03:15, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com>> wrote:
>
> On 2/8/2020 3:52 AM, Klaas de Waal wrote:
> > Hi faginbagin,
> >
> > On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>> wrote:
> >
> >     On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> >     > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com <mailto:mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>>> wrote:
> >     >
> >     >     I ask because I can't get mythtv-setup to find any channels. I first
> >     >     tried a scan before running mythfilldatabase as mentioned in this email
> >     >     thread on mythtv-users:
> >     >     http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> >     >
> >     >     I just tried another channel scan after running mythfilldatabase.
> >     >     mythfilldatabase added lots of channels and listings to the database,
> >     >     but the channel data looked incomplete. So I did a full channel scan,
> >     >     which said it was scanning for ATSC channels, not QAM, and found no
> >     >     channels. I can't find any channel scan settings that distinguish
> >     >     between ATSC and QAM.
> >     >
> >     >     Not surprisingly, the channel data still looks wrong. For example, all
> >     >     of the channel records have a NULL mplexid and the dtv_multiplex table
> >     >     is empty. Has tuning a clear qam channel changed so much that channel
> >     >     records don't need to point to valid dtv_multiplex records any more?
> >     >
> >     >     FWIW, both "General -> Locale Settings -> Channel frequency table" and
> >     >     "Video Sources -> MyVideoSource -> Channel frequency table" are set to
> >     >     us-cable and the listings grabber is "Multinational (Schedules Direct
> >     >     JSON web services w..." and the listings data looks correct.
> >     >
> >     >
> >     > It is interesting to know which version of MythTV you are running. There are lots of changes in the latest master / v31 compared with previous versions.
> >     > You might have missed the Scan Type/ Full Scan settings, see the end of this thread:
> >     > https://forum.mythtv.org/viewtopic.php?f=36&t=3567  
> >     > Also described here:
> >     > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2  
> >     >
> >     >
> >     >
> >     > _______________________________________________
> >     > mythtv-dev mailing list
> >     > mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org> <mailto:mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org>>
> >     > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> >     > http://wiki.mythtv.org/Mailing_List_etiquette
> >     > MythTV Forums: https://forum.mythtv.org
> >     >
> >
> >     Hi Klaas,
> >
> >     My apologies for not mentioning the version, it's v31 from the ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro is lubuntu 19.10 with the lxqt desktop.
> >
> >     Both the forum thread and images got me on the right track. Instead of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256 channels. It's still in progress. I'll report back once I've done further testing.
> >
> >     But I must say the new settings UI is quite confusing and a departure from how other parts of mythtv's UI work. It hadn't occurred to me that right arrow would do something different than OK. Also, that left arrow behaves differently than ESC.
> >
> >     Another point of confusion: after navigating to the scan type page and changing the frequency table and modulation to cable and qam-256, followed by pressing ESC and being asked if I want to save my changes, and saying yes, why am I kicked out of not only the scan type page, but also out of the channel scan page? And when I go back into the scan type page, why are the frequency table and modulation settings back to the original broadcast and 8-vsb settings? Maybe something can be done to explain these points in the help text and the wiki?
> >
> >     And, if I've specified us-cable as the channel frequency table in the channel scan page and also in the Global -> Locale page, why does the frequency table in the scan type page always revert to Broadcast, not to mention 8-vsb modulation? Seems to me there's room for improvement in setting more reasonable defaults based on the us-cable setting as well as saving the settings the user specifies?
> >
> >     Food for thought?
> >
> >
> > I am glad that the scanning does work for you now. 
> > About your painful experiences with the GUI, you have made a good summary of what can be improved. It is largely in line with what I have in mind. For the time being I have documented the way it works in the WIki in  https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people to actually read all of this, But it is there for when desperation sets in....
> > Interesting is also that most of the changes in v31 are "under the hood", the GUI is not really changed since v30 except for a few new options.
>
> Hi Klaas,
>
> I haven't used mythtv-setup to do channel scans in years. My cable company has only made incremental changes over the years, most of which haven't been important to me. When they do make a change I care about, I've used hdhomerun_config to scan for changes, then crafted sql inserts and updates to the channel and dtv_multiplex tables. I tend to upgrade mythtv with every other release, e.g. 0.27 to 29 and now to 31. And I didn't need to spend much time in mythtv-setup to get 29 configured for my needs.
>
> Now I'm getting ready to upgrade to 31 and find it necessary to re-learn how to setup a video source because of the elimination of the DataDirect video source. So, I figured it was also time to see how channel scanning works these days. I had hoped, since the channel table was populated by the new Schedules Direct video source, that maybe the channel scan would be able to update those channel records. But it didn't happen, at least not in my case. Instead, it just added all new channel records for those found in the scan.
>
> In my case, the channel scan only found 68 channels when there are actually just over 100 channels. Maybe increasing timeouts would help. But if I've still got to populate the xmltvid columns, then I think I'm better off starting with my production system's channel and dtv_multiplex tables and using the switch_to_json.py script Bill Meek sent me to automate conversion from the old to the new xmltvids.
>
> Many Thanks for your help in getting channel scanning working.
> _______________________________________________
>
>
> Thanks for testing on USA cable.  I appreciate getting feedback on this since I can test only on DVB sources. I have made changes over time that also can influence the ATSC behaviour but there has no serious regression testing been done on this.

I'd be happy to help.

> About finding  only 68 channels when there are actually just over 100 channels.
> Could you try with larger timeouts to see if that makes a difference? In v31 the default timeout values have increased from 1000/3000 to 3000/6000 but maybe even bigger timeouts are needed. Note that this new default is only for new capture cards, the timeout values of existing capture card entries are not changed.

I have tested with larger timeouts, 6000/9000. No change and I'm pretty sure timeouts aren't the issue. However, I made some changes to the tuner and network environment to rule out some questionable things. I was using an HDHomeRun DUAL Model: HDHR3-US tuner and it was separated from the backend by 2 gigabit switches and a gigabit router. Now I'm using an HDHomeRun CONNECT Model: HDHR4-2US and it's on the same gigabit switch as the backend.

> It could also be that there are frequencies used that are not being tested with a channel scan. The way it works is that a predefined list of frequencies is scanned but maybe that does not cover all frequencies.
> To figure this out, could you provide me with a list of channels as found by hdhomerun_config and also the output of a mythtv-setup channel scan with the "-v chanscan" option? E.g.  "mythtv-setup -v chanscan 2>&1 | tee ms-20200210-2033.log".

I have some data that may or may not interest you. If not, I'd be happy to perform whatever tests you want. I blew away the database and started over with a backup that I made before running mythfilldatabase. I changed the "capturecard" to the newer and closer HDHR4. I then ran a channel scan with the default 3000/6000 timeouts as well as the default scanning options like "Complete scan data required" enabled. Same results as before, Only 68 channels added, although I can see many more were found looking at the logs, 399 to be exact. Most are encrypted, but many aren't. They weren't added, presumably because they didn't meet the "Complete scan data required". I have the output from mythtv-setup -v general,chanscan (but not the stderr output which only complained about XDG_RUNTIME_DIR) and I'm pretty sure the missing channels were found, but were missing callsigns, for example. So I repeated the scan with only "Unencrypted only" enabled. When the scan completed, I answered in the affirmative to all prompts about adding or updating channels. Now there are 150 channels in the table. Some are duplicates my cable provider broadcasts on different frequencies, but others are unique channels that I'm recording on my production system. Also missing are the channel records that were added by mythfilldatabase and that I blew away when I restored an earlier backup. That's the reason why you may not want what I have so far.

As for an hdhomerun_config channel scan. I did one on Jan 12 on my production system, although I don't recall which HDHR tuner I used, probably the old one. The output clearly shows hdhomerun_config found call signs that mythtv-setup didn't. You may want a more recent scan, just in case my cable provider has changed something.

> About creating new channels vs. updating existing channels. 
> The v31 code does try to update existing channels more than before so that non-scanned channel information like xmltvid is preserved across scans.
> If I understand you correct then you have tested with a channel list created by/from Schedules Direct and then do a mythtv-setup channel scan.
> The expected/wanted behaviour is that the channels in the database are updated, but the actual behaviour is that mythtv-setup creates new channels.
> Correct?

Yes that's correct, but I blew away the database and logs with the schedules direct channels and the scan done at the time.

> To figure if the expected behaviour can be realized I need a dump of the tables channel and dtv_multiplex as created by/from SchedulesDirect and also as created by mythtv-setup. 
>
> N.B. the default Channel frequency table for ATSC "Full Scan" is now obtained from the Video Source table value or, if that is "default", from the General / Locale settings table value, in master and in fixes/31. Of course it might take a few days before this is in a repository near you.

Sounds good, it will be nice not to have to drill down to the "Scan Type" page to make those changes. Are you also setting the default modulation as well?

> Thanks,
> Klaas.

If time permits, I will redo things: first running mythfilldatabase to populate the channel table with schedules direct channel data, then an hdhomerun_config scan, followed by a mythtv-setup scan, in the hopes of giving you something to look at tomorrow. But if time doesn't permit, I'll send the data I have now later tonight. In either case, I'll send the data offlist.

One other detail. I have since edited the sqlite database from schedules direct to unselect channels I can't get because they're encrypted. I assume it's a non-issue. If it is, I did save a copy of the original schedules direct database.

More offline...
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Has anyone tested channel scanning in a us-cable Clear QAM environment? [ In reply to ]
On Tue, 11 Feb 2020 at 01:22, faginbagin <mythtv@hbuus.com> wrote:

> On 2/10/2020 2:53 PM, Klaas de Waal wrote:
> > Hi faginbagin,
> >
> > On Mon, 10 Feb 2020 at 03:15, faginbagin <mythtv@hbuus.com <mailto:
> mythtv@hbuus.com>> wrote:
> >
> > On 2/8/2020 3:52 AM, Klaas de Waal wrote:
> > > Hi faginbagin,
> > >
> > > On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv@hbuus.com <mailto:
> mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>>
> wrote:
> > >
> > > On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> > > > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv@hbuus.com
> <mailto:mythtv@hbuus.com> <mailto:mythtv@hbuus.com <mailto:
> mythtv@hbuus.com>> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>
> <mailto:mythtv@hbuus.com <mailto:mythtv@hbuus.com>>>> wrote:
> > > >
> > > > I ask because I can't get mythtv-setup to find any
> channels. I first
> > > > tried a scan before running mythfilldatabase as
> mentioned in this email
> > > > thread on mythtv-users:
> > > >
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> > > >
> > > > I just tried another channel scan after running
> mythfilldatabase.
> > > > mythfilldatabase added lots of channels and listings to
> the database,
> > > > but the channel data looked incomplete. So I did a full
> channel scan,
> > > > which said it was scanning for ATSC channels, not QAM,
> and found no
> > > > channels. I can't find any channel scan settings that
> distinguish
> > > > between ATSC and QAM.
> > > >
> > > > Not surprisingly, the channel data still looks wrong.
> For example, all
> > > > of the channel records have a NULL mplexid and the
> dtv_multiplex table
> > > > is empty. Has tuning a clear qam channel changed so much
> that channel
> > > > records don't need to point to valid dtv_multiplex
> records any more?
> > > >
> > > > FWIW, both "General -> Locale Settings -> Channel
> frequency table" and
> > > > "Video Sources -> MyVideoSource -> Channel frequency
> table" are set to
> > > > us-cable and the listings grabber is "Multinational
> (Schedules Direct
> > > > JSON web services w..." and the listings data looks
> correct.
> > > >
> > > >
> > > > It is interesting to know which version of MythTV you are
> running. There are lots of changes in the latest master / v31 compared with
> previous versions.
> > > > You might have missed the Scan Type/ Full Scan settings, see
> the end of this thread:
> > > > https://forum.mythtv.org/viewtopic.php?f=36&t=3567
> > > > Also described here:
> > > > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > mythtv-dev mailing list
> > > > mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org>
> <mailto:mythtv-dev@mythtv.org <mailto:mythtv-dev@mythtv.org>>
> > > > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > > > http://wiki.mythtv.org/Mailing_List_etiquette
> > > > MythTV Forums: https://forum.mythtv.org
> > > >
> > >
> > > Hi Klaas,
> > >
> > > My apologies for not mentioning the version, it's v31 from the
> ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro
> is lubuntu 19.10 with the lxqt desktop.
> > >
> > > Both the forum thread and images got me on the right track.
> Instead of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256
> channels. It's still in progress. I'll report back once I've done further
> testing.
> > >
> > > But I must say the new settings UI is quite confusing and a
> departure from how other parts of mythtv's UI work. It hadn't occurred to
> me that right arrow would do something different than OK. Also, that left
> arrow behaves differently than ESC.
> > >
> > > Another point of confusion: after navigating to the scan type
> page and changing the frequency table and modulation to cable and qam-256,
> followed by pressing ESC and being asked if I want to save my changes, and
> saying yes, why am I kicked out of not only the scan type page, but also
> out of the channel scan page? And when I go back into the scan type page,
> why are the frequency table and modulation settings back to the original
> broadcast and 8-vsb settings? Maybe something can be done to explain these
> points in the help text and the wiki?
> > >
> > > And, if I've specified us-cable as the channel frequency table
> in the channel scan page and also in the Global -> Locale page, why does
> the frequency table in the scan type page always revert to Broadcast, not
> to mention 8-vsb modulation? Seems to me there's room for improvement in
> setting more reasonable defaults based on the us-cable setting as well as
> saving the settings the user specifies?
> > >
> > > Food for thought?
> > >
> > >
> > > I am glad that the scanning does work for you now.
> > > About your painful experiences with the GUI, you have made a good
> summary of what can be improved. It is largely in line with what I have in
> mind. For the time being I have documented the way it works in the WIki in
> https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people
> to actually read all of this, But it is there for when desperation sets
> in....
> > > Interesting is also that most of the changes in v31 are "under the
> hood", the GUI is not really changed since v30 except for a few new options.
> >
> > Hi Klaas,
> >
> > I haven't used mythtv-setup to do channel scans in years. My cable
> company has only made incremental changes over the years, most of which
> haven't been important to me. When they do make a change I care about, I've
> used hdhomerun_config to scan for changes, then crafted sql inserts and
> updates to the channel and dtv_multiplex tables. I tend to upgrade mythtv
> with every other release, e.g. 0.27 to 29 and now to 31. And I didn't need
> to spend much time in mythtv-setup to get 29 configured for my needs.
> >
> > Now I'm getting ready to upgrade to 31 and find it necessary to
> re-learn how to setup a video source because of the elimination of the
> DataDirect video source. So, I figured it was also time to see how channel
> scanning works these days. I had hoped, since the channel table was
> populated by the new Schedules Direct video source, that maybe the channel
> scan would be able to update those channel records. But it didn't happen,
> at least not in my case. Instead, it just added all new channel records for
> those found in the scan.
> >
> > In my case, the channel scan only found 68 channels when there are
> actually just over 100 channels. Maybe increasing timeouts would help. But
> if I've still got to populate the xmltvid columns, then I think I'm better
> off starting with my production system's channel and dtv_multiplex tables
> and using the switch_to_json.py script Bill Meek sent me to automate
> conversion from the old to the new xmltvids.
> >
> > Many Thanks for your help in getting channel scanning working.
> > _______________________________________________
> >
> >
> > Thanks for testing on USA cable. I appreciate getting feedback on this
> since I can test only on DVB sources. I have made changes over time that
> also can influence the ATSC behaviour but there has no serious regression
> testing been done on this.
>
> I'd be happy to help.
>
> > About finding only 68 channels when there are actually just over 100
> channels.
> > Could you try with larger timeouts to see if that makes a difference? In
> v31 the default timeout values have increased from 1000/3000 to 3000/6000
> but maybe even bigger timeouts are needed. Note that this new default is
> only for new capture cards, the timeout values of existing capture card
> entries are not changed.
>
> I have tested with larger timeouts, 6000/9000. No change and I'm pretty
> sure timeouts aren't the issue. However, I made some changes to the tuner
> and network environment to rule out some questionable things. I was using
> an HDHomeRun DUAL Model: HDHR3-US tuner and it was separated from the
> backend by 2 gigabit switches and a gigabit router. Now I'm using an
> HDHomeRun CONNECT Model: HDHR4-2US and it's on the same gigabit switch as
> the backend.
>
> > It could also be that there are frequencies used that are not being
> tested with a channel scan. The way it works is that a predefined list of
> frequencies is scanned but maybe that does not cover all frequencies.
> > To figure this out, could you provide me with a list of channels as
> found by hdhomerun_config and also the output of a mythtv-setup channel
> scan with the "-v chanscan" option? E.g. "mythtv-setup -v chanscan 2>&1 |
> tee ms-20200210-2033.log".
>
> I have some data that may or may not interest you. If not, I'd be happy to
> perform whatever tests you want. I blew away the database and started over
> with a backup that I made before running mythfilldatabase. I changed the
> "capturecard" to the newer and closer HDHR4. I then ran a channel scan with
> the default 3000/6000 timeouts as well as the default scanning options like
> "Complete scan data required" enabled. Same results as before, Only 68
> channels added, although I can see many more were found looking at the
> logs, 399 to be exact. Most are encrypted, but many aren't. They weren't
> added, presumably because they didn't meet the "Complete scan data
> required". I have the output from mythtv-setup -v general,chanscan (but not
> the stderr output which only complained about XDG_RUNTIME_DIR) and I'm
> pretty sure the missing channels were found, but were missing callsigns,
> for example. So I repeated the scan with only "Unencrypted only" enabled.
> When the scan completed, I answered in the affirmative to all prompts about
> adding or updating channels. Now there are 150 channels in the table. Some
> are duplicates my cable provider broadcasts on different frequencies, but
> others are unique channels that I'm recording on my production system. Also
> missing are the channel records that were added by mythfilldatabase and
> that I blew away when I restored an earlier backup. That's the reason why
> you may not want what I have so far.
>
> As for an hdhomerun_config channel scan. I did one on Jan 12 on my
> production system, although I don't recall which HDHR tuner I used,
> probably the old one. The output clearly shows hdhomerun_config found call
> signs that mythtv-setup didn't. You may want a more recent scan, just in
> case my cable provider has changed something.
>
> > About creating new channels vs. updating existing channels.
> > The v31 code does try to update existing channels more than before so
> that non-scanned channel information like xmltvid is preserved across scans.
> > If I understand you correct then you have tested with a channel list
> created by/from Schedules Direct and then do a mythtv-setup channel scan.
> > The expected/wanted behaviour is that the channels in the database are
> updated, but the actual behaviour is that mythtv-setup creates new channels.
> > Correct?
>
> Yes that's correct, but I blew away the database and logs with the
> schedules direct channels and the scan done at the time.
>
> > To figure if the expected behaviour can be realized I need a dump of the
> tables channel and dtv_multiplex as created by/from SchedulesDirect and
> also as created by mythtv-setup.
> >
> > N.B. the default Channel frequency table for ATSC "Full Scan" is now
> obtained from the Video Source table value or, if that is "default", from
> the General / Locale settings table value, in master and in fixes/31. Of
> course it might take a few days before this is in a repository near you.
>
> Sounds good, it will be nice not to have to drill down to the "Scan Type"
> page to make those changes. Are you also setting the default modulation as
> well?
>
> > Thanks,
> > Klaas.
>
> If time permits, I will redo things: first running mythfilldatabase to
> populate the channel table with schedules direct channel data, then an
> hdhomerun_config scan, followed by a mythtv-setup scan, in the hopes of
> giving you something to look at tomorrow. But if time doesn't permit, I'll
> send the data I have now later tonight. In either case, I'll send the data
> offlist.
>
> One other detail. I have since edited the sqlite database from schedules
> direct to unselect channels I can't get because they're encrypted. I assume
> it's a non-issue. If it is, I did save a copy of the original schedules
> direct database.
>
> More offline...
>
> For all reading this thread, the ATSC Clear QAM channel scanning is fixed
now and the fixes have been committed in the mythtv master today.
Thanks a lot to FaginBagin who has done all the testing and who has
provided me with many gigabytes of captured ATSC streams.

Thanks,
Klaas.