Mailing List Archive

On Sun, Nov 19, 2023 at 12:12?AM Ross Boylan
<rossboylan@stanfordalumni.org> wrote:

> All subchannels for 4 used to be on the same frequency, I think, but
> now they aren't. Some of them have changed callsign as well.

This sounds like a reshuffle to provide for an
ATSC 3.0 lighthouse. The (soon to be) lighthouse
transmitter will arrange to migrate their existing
subchannels to other station owners transmitter
(sometimes having various subchannels on
different transmitters in order to share the
coverage and/or bandwidth).

> I'm concerned about how this will interact with
> 1. SchedulesDirect: do I need a new xmltvid for the new channel? Is
> that automatically updated from SD, or do I need to do something?

"It depends". The upstream of SD will sometimes
reuse the same XMLTV, and sometimes assign a
new one. It will depend, among other things, on
how the station owner is dealing with the change,
and how the (likely) new transmitter is marking
their subchannels (the FCC has some regs about
certain characteristics which can be a forcing
function).

> 2. Existing Recordings: If I delete a virtual channel, like 44.5, or
> change the callsign I assume the old recordings will either have
> nowhere to point for the channel or will have the wrong idea about the
> callsign (and icon) of the station.

In the current releases MythTV will retain existing
channels as long as there is a reference (as long
as you do not try to directly modify the database
outside of the MythTV programs themselves which
can disrupt the business logic). While this retention
can be good, it can also be a bit confusing as the
old WZZZ and the new WZZZ may not be exactly
the same in certain characteristics, even if they
both seem to be WZZZ. For most use cases you
will probably not notice/care, but most does not
mean all.

As you watch and then delete your time shifted
content the lingering issues will naturally
disappear (as will, as I recall, those existing
retained channel definitions marked as deleted).

> 3. Existing Recording Rules: what happens to them? People have
> reported that, at least for rules that are "record on this channel"
> things may stop working.

Correct. Record on "this" channel means exactly
that, *this* channel. The match (as I recall) is based
on callsign (so if the callsign changes as it might it
will no longer match). In general one should try to
avoid using this channel filters when possible (for
some programs, selecting record only new episodes
or record all episodes and retain a small number,
may work better presuming your source guide data
is good and you are consuming it appropriately
with current versions of MythTV). I removed
"this channel" filters from all my rules some time
ago (although my content consumption is going
to be different from yours, so you may still want
or need "this channel").

> Like others I have found rescanning from mythtv-setup to be pretty
> cumbersome; in particular I recall it tends to recreate many channels
> I deliberately deleted.

For some use cases, marking them not visible
may be better than deleting (as depending on what
options you use when loading guide data they
can end up coming back).

In some cases if you are in the US (where
trying to record multiple muxes is rare) and
are using only recent HDHRs and Schedules Direct
one can do a bit better by using a 3rd party
application to update the stations based on
the SD data (if you are also using the grabber
that provides the get-lineup data) and then
tune using virtual channel number(s). This
may require SD to update their lineup when
things change (although the HDHR itself
can be asked to detect channels and find
the new frequency/program for channel 4.1),
which may require opening a ticket with them
if no one else gets there first, and to disable
mythfilldatabase from adding/deleting
stations (two cooks will always spoil the stew).
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sat, 18 Nov 2023 16:11:17 -0800, you wrote:

>Even though the big repacking is done, it seems channels just can't
>sit still. I've written a program to deal with it, but am now getting
>nervous about using it. I'm in the US, get signals OTA via HDHomeRun,
>and use Schedules Direct.
>
>For example, virtual channel 4.1 changed frequency and program #. I
>was going to go in and update the associated mplexid and program # in
>the channel table.
>
>All subchannels for 4 used to be on the same frequency, I think, but
>now they aren't. Some of them have changed callsign as well.
>
>44.5 has disappeared entirely.
>
>I'm concerned about how this will interact with
>1. SchedulesDirect: do I need a new xmltvid for the new channel? Is
>that automatically updated from SD, or do I need to do something?
>2. Existing Recordings: If I delete a virtual channel, like 44.5, or
>change the callsign I assume the old recordings will either have
>nowhere to point for the channel or will have the wrong idea about the
>callsign (and icon) of the station.
>Since I create a new channel id to reflect the new configuration, and
>leave the old one in place. E.g., 4.3 in my channel table points to
>the old frequency and callsign and has chanid 10403, Should I leave
>that as is, and created an entry with chanid (e.g.) 20403 for the new
>configuration?

Virtual channels are simply channels - there is no difference between
main channels and virtual channels except the name and number. Each
is just a group of streams of packets broadcast on a multiplex. In
DVB-T regions, there is no differentiation made - it is just done in
ATSC, probably for political reasons.

There is a fairly recent new feature that keeps old deleted channels
around (marked as deleted) until all the recordings referring to the
channel are deleted. So after you create a new channel under a new
chanid, use mythtv-setup to "delete" the old channel, which will stop
it from being used for new recordings or the Guide/EPG data, but will
retain it for showing which channel was used for old recordings. To
see which channels are deleted, use SQL and select on the
channel.deleted field, which is a timestamp or NULL. Any recordings
before that timestamp match against the deleted channel, any
afterwards match the new channel, so you can have deleted and current
channels with the same callsign and/or xmltvid. So you can have
multiple deleted channels with the same callsign. The chanid field is
a unique key so each version of a channel has to have a different
chanid. This is enforced by MySQL/MariaDB.

>3. Existing Recording Rules: what happens to them? People have
>reported that, at least for rules that are "record on this channel"
>things may stop working.

When channels change, the recording rules are not updated to match,
unfortunately. In the record table, there is a field called "station"
which is the callsign used to match against a channel. When channels
change callsign, you need to manually run SQL to change the matching
record.station fields to the new callsign. When I do this, I make my
SQL update both the record.chanid and record.station fields, although
I believe the record.chanid field is not actually used for matching
anything.

>Like others I have found rescanning from mythtv-setup to be pretty
>cumbersome; in particular I recall it tends to recreate many channels
>I deliberately deleted.

Yes, some people have written scripts to delete all these channels
again whenever they get re-created. The script needs to be manually
altered from time to time. It normally just sets the channel.visible
flag to 0, so the channel will not be re-created on each new scan but
will not be used for anything. If you do not use a script, it is
better to just set channels to not visible rather than deleting them.

>Virtually every conceivable pattern seems to have occurred: channels
>that weren't there get added; those that were there get dropped; any
>of frequency, program number and call sign change for a given virtual
>channel; and sometimes a given channel/callsign jumps to a different
>virtual channel.

Channel numbers (channel.channum in DVB-S/T and
channel.atsc_major_chan/channel.atsc_minor_chan in ATSC) are not used
for anything except humans to select a channel. The things that
matter are channel.chanid (which is used internally and not intended
for human use), channel.callsign and channel.xmltvid. The
channel.name data is not used for matching the EPG data, although I
think it may be used to originally match xmltvid values to channels
when the xmltvids are created.

>Part of the problem is that what seems like a simple, unitary
>concept, a "channel", actually has many elements, which may not change
>in sync: the major and minor channel number (virtual channel), the
>frequency, the program #, call sign, xmltvid, and chanid as used in
>the channel table (among other places). One post suggested that for
>purposes of "record this channel only" the sourceid was also part of
>the identifier.

Yes, the channel updating procedure in MythTV is still not capable
enough to handle all these problems. Since the introduction of
deleted channels it is much better than it was. But more work is
still needed.

"Record this channel only" will match on the callsign/station values,
so if the same channel (with the same callsign) is available on
multiple sources (eg free-to-air, satellite and cable), then MythTV
will record from any matching channel on any source. You set channel
priorities to tell which of the matching channels to prefer, so if you
get a channel in HD on FTA, you would give the FTA channel a higher
priority. If there were insufficient tuners to record from the
highest priority channel, a lower priority one might get used - it
depends on how much higher you set the channel priority compared to
other sources of priority used by the scheduler.

>Thanks for any guidance.
>Ross
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Thank you, Stephen and Gary, for all the info. If I don't run a
rescan or use my external program to make modifications, I'm still not
clear on what happens.

I have been assuming that the program guide, backed by info from SD,
is accurate (except for glitches, and perhaps delays catching up to
channel redefinitions). Accurate means the programs listed on, e.g.,
4.3 are what I'd actually see if I tuned to 4.3. Less importantly, it
means the callsign and icon are accurate. But is any of that true?
For example, if the xmltvid is not updated, and if that's the key for
matching SD, then I'd be getting a listing for something that might be
showing on a different channel, or not at all. @Gary says that
sometimes SD will continue to use the same xmltvid, in which case a
mismatch on that would not be a problem. But that also means that
sometimes it changes, and then what happens in mythtv?

The interpretation of "same channel" as "same callsign" explains why
sometimes I will try to limit recording to, e.g., 11.1, and I end up
getting double recordings on 11.1 and 11.3: they have the same
callsign. (Actually, slightly different: "KNTV HD" vs "KNTV-HD". But
I think I've had this problem with that pair).

BTW, the changes to virtual channel 4 are definitely part of an ATSC
3.0 move. And 44.5 already has a date/time in channel.deleted. I
used the Services API Channel/RemoveDBChannel to do that, which must
mean it didn't get deleted on its own or via an update from SD.

Ross
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sat, 18 Nov 2023 20:55:37 -0800, you wrote:

>Thank you, Stephen and Gary, for all the info. If I don't run a
>rescan or use my external program to make modifications, I'm still not
>clear on what happens.
>
>I have been assuming that the program guide, backed by info from SD,
>is accurate (except for glitches, and perhaps delays catching up to
>channel redefinitions). Accurate means the programs listed on, e.g.,
>4.3 are what I'd actually see if I tuned to 4.3. Less importantly, it
>means the callsign and icon are accurate. But is any of that true?
>For example, if the xmltvid is not updated, and if that's the key for
>matching SD, then I'd be getting a listing for something that might be
>showing on a different channel, or not at all. @Gary says that
>sometimes SD will continue to use the same xmltvid, in which case a
>mismatch on that would not be a problem. But that also means that
>sometimes it changes, and then what happens in mythtv?

Sorry, I can not help with SchedulesDirect as they do not do New
Zealand EPG. I believe it is possible to get channels created from
the XMLTV EPG data, rather than from a channel scan. So should you be
doing that instead of scanning?

>The interpretation of "same channel" as "same callsign" explains why
>sometimes I will try to limit recording to, e.g., 11.1, and I end up
>getting double recordings on 11.1 and 11.3: they have the same
>callsign. (Actually, slightly different: "KNTV HD" vs "KNTV-HD". But
>I think I've had this problem with that pair).

With callsigns there is no such thing as "slightly different" - any
difference at all means it is a different channel. There is no
matching for "similar" callsigns.

I have a channel that is on two sources where I often see that the
upcoming recordings list is showing that it will record from both
sources. But in fact it only records from the higher priority source
(free-to-air DVB-T). I think this is a bug, but not a bad one as it
is only a display problem. And very occasionally, I have had both
channels recorded. I think, but have not confirmed, that it has
happened when the DVB-T tuner was taking a long time to tune, so the
recording from there was not marked as started, and the DVB-S2
recording was then also started.

>BTW, the changes to virtual channel 4 are definitely part of an ATSC
>3.0 move. And 44.5 already has a date/time in channel.deleted. I
>used the Services API Channel/RemoveDBChannel to do that, which must
>mean it didn't get deleted on its own or via an update from SD.

Any channel deletion using the proper MythTV tools (the API or
mythtv-setup) will mark the channel as deleted if there are any
recordings that match the channel. If there are no matching
recordings, I am not sure of the exact process - it may just be marked
as deleted in the same way, or it may be completely removed from the
channel table. In any case, there seems to be a housekeeping task
that runs regularly (daily?) that will completely remove old channels
when there are no more matching recordings.

The channel scan process now uses the same delete channel procedure as
manual deletions - it will mark channels as deleted if they disappear
from their source.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

FYI, a while ago the possibility to recover XMLTVID from deleted channels
was added, see https://www.mythtv.org/wiki/Channel_Scanning#Restore_Data
Never tested this with ATSC, but it might be useful.

Klaas.


On Sun, 19 Nov 2023 at 06:21, Stephen Worthington <stephen_agent@jsw.gen.nz>
wrote:

> On Sat, 18 Nov 2023 20:55:37 -0800, you wrote:
>
> >Thank you, Stephen and Gary, for all the info. If I don't run a
> >rescan or use my external program to make modifications, I'm still not
> >clear on what happens.
> >
> >I have been assuming that the program guide, backed by info from SD,
> >is accurate (except for glitches, and perhaps delays catching up to
> >channel redefinitions). Accurate means the programs listed on, e.g.,
> >4.3 are what I'd actually see if I tuned to 4.3. Less importantly, it
> >means the callsign and icon are accurate. But is any of that true?
> >For example, if the xmltvid is not updated, and if that's the key for
> >matching SD, then I'd be getting a listing for something that might be
> >showing on a different channel, or not at all. @Gary says that
> >sometimes SD will continue to use the same xmltvid, in which case a
> >mismatch on that would not be a problem. But that also means that
> >sometimes it changes, and then what happens in mythtv?
>
> Sorry, I can not help with SchedulesDirect as they do not do New
> Zealand EPG. I believe it is possible to get channels created from
> the XMLTV EPG data, rather than from a channel scan. So should you be
> doing that instead of scanning?
>
> >The interpretation of "same channel" as "same callsign" explains why
> >sometimes I will try to limit recording to, e.g., 11.1, and I end up
> >getting double recordings on 11.1 and 11.3: they have the same
> >callsign. (Actually, slightly different: "KNTV HD" vs "KNTV-HD". But
> >I think I've had this problem with that pair).
>
> With callsigns there is no such thing as "slightly different" - any
> difference at all means it is a different channel. There is no
> matching for "similar" callsigns.
>
> I have a channel that is on two sources where I often see that the
> upcoming recordings list is showing that it will record from both
> sources. But in fact it only records from the higher priority source
> (free-to-air DVB-T). I think this is a bug, but not a bad one as it
> is only a display problem. And very occasionally, I have had both
> channels recorded. I think, but have not confirmed, that it has
> happened when the DVB-T tuner was taking a long time to tune, so the
> recording from there was not marked as started, and the DVB-S2
> recording was then also started.
>
> >BTW, the changes to virtual channel 4 are definitely part of an ATSC
> >3.0 move. And 44.5 already has a date/time in channel.deleted. I
> >used the Services API Channel/RemoveDBChannel to do that, which must
> >mean it didn't get deleted on its own or via an update from SD.
>
> Any channel deletion using the proper MythTV tools (the API or
> mythtv-setup) will mark the channel as deleted if there are any
> recordings that match the channel. If there are no matching
> recordings, I am not sure of the exact process - it may just be marked
> as deleted in the same way, or it may be completely removed from the
> channel table. In any case, there seems to be a housekeeping task
> that runs regularly (daily?) that will completely remove old channels
> when there are no more matching recordings.
>
> The channel scan process now uses the same delete channel procedure as
> manual deletions - it will mark channels as deleted if they disappear
> from their source.
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>

Ross Boylan <rossboylan@stanfordalumni.org> wrote:

> Even though the big repacking is done, it seems channels just can't
> sit still.

It’s a right p.i.t.a. I agree. To a certain extent we are lucky in the UK in (I think) that “someone” restricts reshuffles to 2 or 3 times a year at most. Still, it’s enough that I long ago gave up manually reordering channels on the TVs after each rescan - SWMBO is a bit of a technophobe and only uses the Myth frontends for watching recordings, so the TVs have to be kept in tune.

Many years ago I wrote a script to reset channels after a rescan https://www.mythtv.org/wiki/Database_editing_script
The page is a bit out of date now, for example where it “dots” in the config file, that doesn’t work any more as the config file has changed to XML.

It takes a bit of work to maintain, but a lot less than doing it from scratch each time. The technique I use is the delete all channels, rescan all transports, re-run the script and see what’s missing ! Things it can’t match by callsign end up at 10k+ in the channel editor.
So then it’s a matter of working out what’s what (e.g. which channel has changed to a new callsign), finding new XMLTV IDs, and updating the script. It can take a few tries to get it right.
I have a script that greps the XML IDs and callsigns from a grab and then use ${favourite_editor} to search the result :
> tv_grab_zz_sdjson --config-file ${conf}.full --days 0 --output ${chan_file}.xml
> grep 'channel id
> display-name' < ${chan_file}.xml > ${chan_file}.txt

In my grabber script, I grab the XML ides from the database and dynamically build the config file so tv_grab_zz_sdjson just pulls listings for channels I’ve configured.
> mysql --host=localhost --user=mythtv --database=mythconverg --password=‘<password>' \
> -e "select distinct(xmltvid) from channel where visible=1 and useonairguide=0 order by xmltvid ;" |
> grep '[0-9][0-9][0-9][0-9][0-9]' |
> sed -e 's/^/channel=/' >> ${conf}

It’s not pretty or fast, just functional and does the job for something that’s only run once a day.


Simon

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

This is a followup report on the effects of some changes. I took a
channel, 4.1 in San Francisco, that apparently had only a change in
frequency and program #. I had exactly one existing plex at the new
frequency, and I changed mplexid in the relevant row of the channel
table to point to it.

This alone seems to have been enough so that myth could tune to the
channel, which must mean it is either ignoring the program #, which I
think is what channel.serviceid is, or looking harder if it seems not
to work.

And clearly having the correct freqid is not important, though I
eventually changed it and the seriesid to match what I think are the
right values (from the OTA scan of HDHR).

The channel listings I'm getting seem to match what is broadcast.
Although my last scan said the channel was KRON-TV, my listing seems
to identify it as the CW. The internet confirms the station is more
or less owned by The CW.

Ross
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sun, 26 Nov 2023 21:24:00 -0800, you wrote:

>This is a followup report on the effects of some changes. I took a
>channel, 4.1 in San Francisco, that apparently had only a change in
>frequency and program #. I had exactly one existing plex at the new
>frequency, and I changed mplexid in the relevant row of the channel
>table to point to it.
>
>This alone seems to have been enough so that myth could tune to the
>channel, which must mean it is either ignoring the program #, which I
>think is what channel.serviceid is, or looking harder if it seems not
>to work.
>
>And clearly having the correct freqid is not important, though I
>eventually changed it and the seriesid to match what I think are the
>right values (from the OTA scan of HDHR).
>
>The channel listings I'm getting seem to match what is broadcast.
>Although my last scan said the channel was KRON-TV, my listing seems
>to identify it as the CW. The internet confirms the station is more
>or less owned by The CW.
>
>Ross

The freqid field is only used for analogue channels - it is completely
ignored for any digital channel. I believe scans of digital sources
will now set it to NULL for new channels. To find the frequency a
digital channel is on, you have to use SQL like this:

MariaDB [mythconverg]> select frequency from dtv_multiplex where
mplexid=(select mplexid from channel where callsign='TVNZ 1');
+-----------+
| frequency |
+-----------+
| 578000000 |
+-----------+
1 row in set (0.001 sec)

The serviceid values normally stay the same when a channel moves
frequency. They are supposed to be unique across all the channels
that are able to be received at any site, and are usually unique
across an entire TV system. So when moving a channel to another mux,
the serviceid normally does not need to be changed. If a channel is
moving to a mux that already exists in dtv_multiplex, then yes, just
changing the mplexid is usually all that is required.

The serviceid is used to match EIT EPG data to a channel, so the
channel name in the scan data and the channel name in the EPG data (if
present) do not need to match, and occasionally are actually
different. Matching of external EPG data is done on the xmltvid value
for the channel. Often xmltvid values created automatically have the
serviceid as the first part, for example: 1275.dvb.guide.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 11/27/2023 12:24 AM, Ross Boylan wrote:
> This is a followup report on the effects of some changes. I took a
> channel, 4.1 in San Francisco, that apparently had only a change in
> frequency and program #. I had exactly one existing plex at the new
> frequency, and I changed mplexid in the relevant row of the channel
> table to point to it.
>
> This alone seems to have been enough so that myth could tune to the
> channel, which must mean it is either ignoring the program #, which I
> think is what channel.serviceid is, or looking harder if it seems not
> to work.
>
> And clearly having the correct freqid is not important, though I
> eventually changed it and the seriesid to match what I think are the
> right values (from the OTA scan of HDHR).
>
> The channel listings I'm getting seem to match what is broadcast.
> Although my last scan said the channel was KRON-TV, my listing seems
> to identify it as the CW. The internet confirms the station is more
> or less owned by The CW.
>
> Ross
>
What I have observed about the channel and dtv_multiplex tables and
tuning is that if you're tuning an ATSC channel, what matters are the
channel.mplexid (which points to the dtv_multiplex.frequency) and the
channel.atsc_major/minor numbers. If you are tuning a clear QAM channel
(as in unencrypted cable), then the mplexid/frequency and
channel.serviceid are used. The channel.freqid was used to tune analog
channels, but is no longer used. However, it is populated by the channel
scanner to correspond to the analog channel's frequency as provided by
the dtv_multiplex.frequency. Having them in sync comes in handy when
using the HDHR channel scanning tools, since that freqid is what the
HDHR gui displays as the channel.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Mon, Nov 27, 2023 at 12:47?AM Stephen Worthington
<stephen_agent@jsw.gen.nz> wrote:
>
...
> The serviceid values normally stay the same when a channel moves
> frequency. They are supposed to be unique across all the channels
> that are able to be received at any site, and are usually unique
> across an entire TV system. So when moving a channel to another mux,
> the serviceid normally does not need to be changed.

This is definitely not the case in the US with ATSC. The serviceid
appears to have been set from the "program #" used to pick out streams
being broadcast on the same frequency. They are typically low
numbers, like 1-5, for each distinct frequency. So when an individual
subchannel, like virtual 4.1 (I'm not sure if technically 4 is the
virtual channel or 4.1) moves to another frequency it often needs a
different, low, number. But, as I found and others have said, it
seems to be informational only.

Ross
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org