Mailing List Archive

Channel Management Ideas and Plans
Here's a brief description of some of the things relating to channel
management that I hope to work on. I'll likely start on some of them
soon. I'll get to the others when/if I get to them. Please provide
comments or other suggestions here.

Removal of DataDirect Support
=============================

This has been discussed many times before and already announced here.
The intent is to simplify mythfilldatabase and related functionality
by only supporting xmltv.

Remove Some Dependencies on the Channel Table
=============================================

When program entries are primarily read from the recorded table (Watch
Recordings screen) and the oldrecorded table (Previously Recorded
screen), they are augmented with data from the channel table. This is
the reason that channel and videosource entries must be kept in the
database even after they are no longer being used for new recordings.
The intent of the following options is to remove these dependencies on
the channel table so that channels and videosources can be deleted as
soon as they are no longer used.

Option 1. Add the relevant columns to the other tables. The recorded
table would add channum, callsign and channame(*). The oldrecorded
table would add channum and channame.

Option 1a. The same as option 1 but add the new columns to the
recordedprogram table instead of the recorded table. I prefer plain,
option 1 for simplicity and keeping the program and recordedprogram
tables with the same columns.

Option 2. Add some new indication in the channel table for "Deleted"
channels. This could work with the cost being more complexity to
ignore deleted channels in the appropriate places and deciding when to
actually delete an entry or mark it as deleted.

Right now, I prefer option 1 due to its simplicity. One downside is
that other potentially useful data about the channel would be lost
when it's deleted. For example, what if a themer wanted to show the
channel icon in Watch Recordings instead of or in addition to the
preview image?

(*)A couple of other columns from the channel table are pulled in here
but I believe they are only relevant immediately after the program was
recorded. Returning NULLs if the channel is deleted much later should
not cause a problem.

Enhance Channel Visibility with New Values
==========================================

Gary Buhrmaster has a nice script for managing the channel.visible
setting for HDHomeRun Prime users(*)(#). Unfortunately, the script
doesn't and can't know things that only the user knows. Things like
"even though I can receive the channel, I don't want it visible" or
"the channel isn't receivable right now but I know it will be when I
plan to use it".

To solve this problem, I intend to extend the meaning of
channel.visible from simply no and yes values to also include values
for "never" and "always". Gary's script could then be changed to know
not to override those values.

(*)I can envision other, similar scripts and even functionality built
into MythTV for periodically scanning and validating channel
reception. This change would be useful for them too.

(#)Gary also has other scripts that are very useful for MythTV users.
In fact, they are part of the inspiration for this series of changes.

Better Automated Handling of Channel Additions/Deletions
=======================================================

Handling of channels additions and deletions in mythfilldatabase has
always seemed way more complex to me than needed. Perhaps that
because of the DataDirect complication (see above) or I'm just to
naive to understand all of the subtleties. Either way, I hope to
improve on it.

Note that I'm primarily concerning myself with the
mythfilldatabase/xmltv part of this problem for the time being. I
know I can't totally ignore channel scanning for those tuner types
that require it but major changes in the is not something I'm prepared
to take on.

Anyway, I'm looking to add new settings for each videosource to
control channel additions and deletions. For additions, I'm thinking
of a setting with values like "ignore", "add as never visible", "add
as invisible", "add as visible" and "add as always visible". If it's
possible to actually check the reception then also have an option for
"add as detected". For deletions, I'm thinking of a setting with
values like "ignore", "delete", "mark as invisible" and "mark as
never visible".

Better Automated Handling of Channel Updates
============================================

Besides channels getting added and deleted, other channel information
like names, callsigns and icons get updated over time. I don't know
if mythfilldatabasse handles any of that or not. Gary Buhrmaster
(surprise), though, has a script for doing much of this, at least for
tv_grab_zz_sdjson_sqlite users. I'd like to see some of that
functionality get rolled into MythTV proper. I don't know if that
should be mythfilldatabase, mythutil or an entirely new utility at
this point.

One complicating factor is deciding how granularly to control the
updates -- at the videosource level and/or the individual channel
level. Also, which information to allow updates -- purely cosmetic
stuff like name vs. callsign (with scheduling implications) vs. tuning
stuff like frequency and multiplex. Gary's script allows very
granular control which is great for me because I need it. Others
probably don't and just makes things more complex for them.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sun, Jan 27, 2019 at 2:09 PM David Engel <david@istwok.net> wrote:

> Enhance Channel Visibility with New Values
> ==========================================
>
> Gary Buhrmaster has a nice script for managing the channel.visible
> setting for HDHomeRun Prime users(*)(#). Unfortunately, the script
> doesn't and can't know things that only the user knows. Things like
> "even though I can receive the channel, I don't want it visible" or
> "the channel isn't receivable right now but I know it will be when I
> plan to use it".
>
> To solve this problem, I intend to extend the meaning of
> channel.visible from simply no and yes values to also include values
> for "never" and "always". Gary's script could then be changed to know
> not to override those values.
>
> (*)I can envision other, similar scripts and even functionality built
> into MythTV for periodically scanning and validating channel
> reception. This change would be useful for them too.
>
> (#)Gary also has other scripts that are very useful for MythTV users.
> In fact, they are part of the inspiration for this series of changes.
>

I am still with Directv (mostly because they cut my bill in half for the
next 12 months when I went to cancel), and they do "free" weekends about
once per quarter, where some subset of their "premium" (e.g. HBO) channels
are free. I currently have a hacky patch I use to make those channels
visible for a specific date/time range. Whenever
https://www.freepreview.tv/ indicates that such a weekend is coming up, I
add the dates and times to a configuration file, which is used to override
the visible flag for the specified channels.

The patch is hacky enough that I have never intended to commit it, but if
you are re-working the core of how visibility works, it might be useful to
take this scenario into account.

John
Re: Channel Management Ideas and Plans [ In reply to ]
On January 27, 2019 4:09:39 PM EST, David Engel <david@istwok.net> wrote:
>Here's a brief description of some of the things relating to channel
>management that I hope to work on. I'll likely start on some of them
>soon. I'll get to the others when/if I get to them. Please provide
>comments or other suggestions here.
>
>Removal of DataDirect Support
>=============================
>
>This has been discussed many times before and already announced here.
>The intent is to simplify mythfilldatabase and related functionality
>by only supporting xmltv.
>
>Remove Some Dependencies on the Channel Table
>=============================================
>
>When program entries are primarily read from the recorded table (Watch
>Recordings screen) and the oldrecorded table (Previously Recorded
>screen), they are augmented with data from the channel table. This is
>the reason that channel and videosource entries must be kept in the
>database even after they are no longer being used for new recordings.
>The intent of the following options is to remove these dependencies on
>the channel table so that channels and videosources can be deleted as
>soon as they are no longer used.
>
>Option 1. Add the relevant columns to the other tables. The recorded
>table would add channum, callsign and channame(*). The oldrecorded
>table would add channum and channame.
>
>Option 1a. The same as option 1 but add the new columns to the
>recordedprogram table instead of the recorded table. I prefer plain,
>option 1 for simplicity and keeping the program and recordedprogram
>tables with the same columns.
>
>Option 2. Add some new indication in the channel table for "Deleted"
>channels. This could work with the cost being more complexity to
>ignore deleted channels in the appropriate places and deciding when to
>actually delete an entry or mark it as deleted.
>
>Right now, I prefer option 1 due to its simplicity. One downside is
>that other potentially useful data about the channel would be lost
>when it's deleted. For example, what if a themer wanted to show the
>channel icon in Watch Recordings instead of or in addition to the
>preview image?
>
>(*)A couple of other columns from the channel table are pulled in here
>but I believe they are only relevant immediately after the program was
>recorded. Returning NULLs if the channel is deleted much later should
>not cause a problem.
>
>Enhance Channel Visibility with New Values
>==========================================
>
>Gary Buhrmaster has a nice script for managing the channel.visible
>setting for HDHomeRun Prime users(*)(#). Unfortunately, the script
>doesn't and can't know things that only the user knows. Things like
>"even though I can receive the channel, I don't want it visible" or
>"the channel isn't receivable right now but I know it will be when I
>plan to use it".
>
>To solve this problem, I intend to extend the meaning of
>channel.visible from simply no and yes values to also include values
>for "never" and "always". Gary's script could then be changed to know
>not to override those values.
>
>(*)I can envision other, similar scripts and even functionality built
>into MythTV for periodically scanning and validating channel
>reception. This change would be useful for them too.
>
>(#)Gary also has other scripts that are very useful for MythTV users.
>In fact, they are part of the inspiration for this series of changes.
>
>Better Automated Handling of Channel Additions/Deletions
>=======================================================
>
>Handling of channels additions and deletions in mythfilldatabase has
>always seemed way more complex to me than needed. Perhaps that
>because of the DataDirect complication (see above) or I'm just to
>naive to understand all of the subtleties. Either way, I hope to
>improve on it.
>
>Note that I'm primarily concerning myself with the
>mythfilldatabase/xmltv part of this problem for the time being. I
>know I can't totally ignore channel scanning for those tuner types
>that require it but major changes in the is not something I'm prepared
>to take on.
>
>Anyway, I'm looking to add new settings for each videosource to
>control channel additions and deletions. For additions, I'm thinking
>of a setting with values like "ignore", "add as never visible", "add
>as invisible", "add as visible" and "add as always visible". If it's
>possible to actually check the reception then also have an option for
>"add as detected". For deletions, I'm thinking of a setting with
>values like "ignore", "delete", "mark as invisible" and "mark as
>never visible".
>
>Better Automated Handling of Channel Updates
>============================================
>
>Besides channels getting added and deleted, other channel information
>like names, callsigns and icons get updated over time. I don't know
>if mythfilldatabasse handles any of that or not. Gary Buhrmaster
>(surprise), though, has a script for doing much of this, at least for
>tv_grab_zz_sdjson_sqlite users. I'd like to see some of that
>functionality get rolled into MythTV proper. I don't know if that
>should be mythfilldatabase, mythutil or an entirely new utility at
>this point.
>
>One complicating factor is deciding how granularly to control the
>updates -- at the videosource level and/or the individual channel
>level. Also, which information to allow updates -- purely cosmetic
>stuff like name vs. callsign (with scheduling implications) vs. tuning
>stuff like frequency and multiplex. Gary's script allows very
>granular control which is great for me because I need it. Others
>probably don't and just makes things more complex for them.
>
>David

Regarding option 1, I assume you aren't suggesting adding callsign to oldrecorded because it already has a station column, which seems to me to be an alias for callsign. Would it make sense to change that column name to callsign to be consistent across the board?
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sun, Jan 27, 2019 at 06:14:05PM -0500, faginbagin wrote:
> On January 27, 2019 4:09:39 PM EST, David Engel <david@istwok.net> wrote:
> >Here's a brief description of some of the things relating to channel
> >management that I hope to work on. I'll likely start on some of them
> >soon. I'll get to the others when/if I get to them. Please provide
> >comments or other suggestions here.
> >
> >Removal of DataDirect Support
> >=============================
> >
> >This has been discussed many times before and already announced here.
> >The intent is to simplify mythfilldatabase and related functionality
> >by only supporting xmltv.
> >
> >Remove Some Dependencies on the Channel Table
> >=============================================
> >
> >When program entries are primarily read from the recorded table (Watch
> >Recordings screen) and the oldrecorded table (Previously Recorded
> >screen), they are augmented with data from the channel table. This is
> >the reason that channel and videosource entries must be kept in the
> >database even after they are no longer being used for new recordings.
> >The intent of the following options is to remove these dependencies on
> >the channel table so that channels and videosources can be deleted as
> >soon as they are no longer used.
> >
> >Option 1. Add the relevant columns to the other tables. The recorded
> >table would add channum, callsign and channame(*). The oldrecorded
> >table would add channum and channame.
> >
> >Option 1a. The same as option 1 but add the new columns to the
> >recordedprogram table instead of the recorded table. I prefer plain,
> >option 1 for simplicity and keeping the program and recordedprogram
> >tables with the same columns.
> >
> >Option 2. Add some new indication in the channel table for "Deleted"
> >channels. This could work with the cost being more complexity to
> >ignore deleted channels in the appropriate places and deciding when to
> >actually delete an entry or mark it as deleted.
> >
> >Right now, I prefer option 1 due to its simplicity. One downside is
> >that other potentially useful data about the channel would be lost
> >when it's deleted. For example, what if a themer wanted to show the
> >channel icon in Watch Recordings instead of or in addition to the
> >preview image?
> >
> >(*)A couple of other columns from the channel table are pulled in here
> >but I believe they are only relevant immediately after the program was
> >recorded. Returning NULLs if the channel is deleted much later should
> >not cause a problem.
> >
> >Enhance Channel Visibility with New Values
> >==========================================
> >
> >Gary Buhrmaster has a nice script for managing the channel.visible
> >setting for HDHomeRun Prime users(*)(#). Unfortunately, the script
> >doesn't and can't know things that only the user knows. Things like
> >"even though I can receive the channel, I don't want it visible" or
> >"the channel isn't receivable right now but I know it will be when I
> >plan to use it".
> >
> >To solve this problem, I intend to extend the meaning of
> >channel.visible from simply no and yes values to also include values
> >for "never" and "always". Gary's script could then be changed to know
> >not to override those values.
> >
> >(*)I can envision other, similar scripts and even functionality built
> >into MythTV for periodically scanning and validating channel
> >reception. This change would be useful for them too.
> >
> >(#)Gary also has other scripts that are very useful for MythTV users.
> >In fact, they are part of the inspiration for this series of changes.
> >
> >Better Automated Handling of Channel Additions/Deletions
> >=======================================================
> >
> >Handling of channels additions and deletions in mythfilldatabase has
> >always seemed way more complex to me than needed. Perhaps that
> >because of the DataDirect complication (see above) or I'm just to
> >naive to understand all of the subtleties. Either way, I hope to
> >improve on it.
> >
> >Note that I'm primarily concerning myself with the
> >mythfilldatabase/xmltv part of this problem for the time being. I
> >know I can't totally ignore channel scanning for those tuner types
> >that require it but major changes in the is not something I'm prepared
> >to take on.
> >
> >Anyway, I'm looking to add new settings for each videosource to
> >control channel additions and deletions. For additions, I'm thinking
> >of a setting with values like "ignore", "add as never visible", "add
> >as invisible", "add as visible" and "add as always visible". If it's
> >possible to actually check the reception then also have an option for
> >"add as detected". For deletions, I'm thinking of a setting with
> >values like "ignore", "delete", "mark as invisible" and "mark as
> >never visible".
> >
> >Better Automated Handling of Channel Updates
> >============================================
> >
> >Besides channels getting added and deleted, other channel information
> >like names, callsigns and icons get updated over time. I don't know
> >if mythfilldatabasse handles any of that or not. Gary Buhrmaster
> >(surprise), though, has a script for doing much of this, at least for
> >tv_grab_zz_sdjson_sqlite users. I'd like to see some of that
> >functionality get rolled into MythTV proper. I don't know if that
> >should be mythfilldatabase, mythutil or an entirely new utility at
> >this point.
> >
> >One complicating factor is deciding how granularly to control the
> >updates -- at the videosource level and/or the individual channel
> >level. Also, which information to allow updates -- purely cosmetic
> >stuff like name vs. callsign (with scheduling implications) vs. tuning
> >stuff like frequency and multiplex. Gary's script allows very
> >granular control which is great for me because I need it. Others
> >probably don't and just makes things more complex for them.
> >
> >David
>
> Regarding option 1, I assume you aren't suggesting adding callsign to oldrecorded because it already has a station column, which seems to me to be an alias for callsign. Would it make sense to change that column name to callsign to be consistent across the board?

You are correct that oldrecorded already has the station column.
That's why I only listed channum and channame for addition to
oldrecorded.

As for renaming station to callsign, we have generally discouraged
such schema changes as gratuitous in the past. The reason is to make
it easier to compare and move data between older and newer schema
versions. The developer who was most vocal about that policy is long,
long, long, gone, though. If the other, current developers want to
revisit that policy, we can.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sun, 27 Jan 2019 15:09:39 -0600, you wrote:

[snip]

An additional complication that you may not have thought about - when
a channel changes its identity (station name, callsign, icon),
obviously the recording rules for that callsign need to be updated
also. But if you are using "power" recording rules, there will often
be SQL stored in the recording rule that will contain a reference to
the callsign and/or the chanid. It would be good if those rules could
at least be flagged for manual fixing, even if an automated fix was
not possible.

In this part of the world, channels are assigned XMLTV IDs by the EPG
grabbing software that creates the xmltv data. Those names are
normally made from the channel name found when scanning for channels.
When the station identity changes, those names also need to be changed
if you are trying to keep things consistent.

Here are the current XMLTV IDs I am using for our free-to-air
channels:

tv1.freeviewnz.tv
tv2.freeviewnz.tv
tv3.freeviewnz.tv
bravo.freeviewnz.tv
maori-tv.freeviewnz.tv
tv1-plus1.freeviewnz.tv
tv2-plus1.freeviewnz.tv
tv3-plus1.freeviewnz.tv
bravo-plus1.freeviewnz.tv
duke.freeviewnz.tv
edge-tv.freeviewnz.tv
prime.freeviewnz.tv
choice.freeviewnz.tv
hgtv.freeviewnz.tv
parliament.freeviewnz.tv
hope-channel.freeviewnz.tv
aljazeera.freeviewnz.tv
te-reo.freeviewnz.tv
firstlight.freeviewnz.tv
tvsn-shopping-channel.freeviewnz.tv
chinese-tv8.freeviewnz.tv
tv29.freeviewnz.tv
tv33.freeviewnz.tv
apna.freeviewnz.tv
rnz-national.freeviewnz.tv
rnz-concert.freeviewnz.tv
basefm.freeviewnz.tv
threelife.freeviewnz.tv

and here are a few channels from our pay-TV satellite provider:

cnbc.sky.co.nz
cnn.sky.co.nz
comedy-central.sky.co.nz
concert-fm.radio.sky.co.nz
country.music.sky.co.nz
countrytv.sky.co.nz
crime-investigation.sky.co.nz

Another way I have thought about for keeping old channel data
available is to simply create an "oldchannel" table. No longer
available channels would be moved from "channel" to "oldchannel".
Occasionally, housekeeping could be done to scan the recordings tables
and check if there are any channels in "oldchannel" that no longer
have matching recordings, and then delete those channels from
"oldchannel". That could be done by an external script, as is done
with find_orphans.py. Or it could be added to find_orphans.py. Using
an oldchannel table greatly reduces the redundant data that might
otherwise need to be stored in the "recorded" table and allows all of
the old data for a channel to be kept. But it complicates the use of
chanid values as they are now a primary key on two tables and need to
be unique over both tables.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sun, Jan 27, 2019 at 03:09:39PM -0600, David Engel wrote:
> Here's a brief description of some of the things relating to channel
> management that I hope to work on. I'll likely start on some of them
> soon. I'll get to the others when/if I get to them. Please provide
> comments or other suggestions here.
>
> Removal of DataDirect Support
> =============================
>
> This has been discussed many times before and already announced here.
> The intent is to simplify mythfilldatabase and related functionality
> by only supporting xmltv.
>
> Remove Some Dependencies on the Channel Table
> =============================================
>
> When program entries are primarily read from the recorded table (Watch
> Recordings screen) and the oldrecorded table (Previously Recorded
> screen), they are augmented with data from the channel table. This is
> the reason that channel and videosource entries must be kept in the
> database even after they are no longer being used for new recordings.
> The intent of the following options is to remove these dependencies on
> the channel table so that channels and videosources can be deleted as
> soon as they are no longer used.
>
> Option 1. Add the relevant columns to the other tables. The recorded
> table would add channum, callsign and channame(*). The oldrecorded
> table would add channum and channame.
>
> Option 1a. The same as option 1 but add the new columns to the
> recordedprogram table instead of the recorded table. I prefer plain,
> option 1 for simplicity and keeping the program and recordedprogram
> tables with the same columns.
>
> Option 2. Add some new indication in the channel table for "Deleted"
> channels. This could work with the cost being more complexity to
> ignore deleted channels in the appropriate places and deciding when to
> actually delete an entry or mark it as deleted.
>
> Right now, I prefer option 1 due to its simplicity. One downside is
> that other potentially useful data about the channel would be lost
> when it's deleted. For example, what if a themer wanted to show the
> channel icon in Watch Recordings instead of or in addition to the
> preview image?
>
> (*)A couple of other columns from the channel table are pulled in here
> but I believe they are only relevant immediately after the program was
> recorded. Returning NULLs if the channel is deleted much later should
> not cause a problem.
>
> Enhance Channel Visibility with New Values
> ==========================================
>
> Gary Buhrmaster has a nice script for managing the channel.visible
> setting for HDHomeRun Prime users(*)(#). Unfortunately, the script
> doesn't and can't know things that only the user knows. Things like
> "even though I can receive the channel, I don't want it visible" or
> "the channel isn't receivable right now but I know it will be when I
> plan to use it".
>
> To solve this problem, I intend to extend the meaning of
> channel.visible from simply no and yes values to also include values
> for "never" and "always". Gary's script could then be changed to know
> not to override those values.
>
> (*)I can envision other, similar scripts and even functionality built
> into MythTV for periodically scanning and validating channel
> reception. This change would be useful for them too.
>
> (#)Gary also has other scripts that are very useful for MythTV users.
> In fact, they are part of the inspiration for this series of changes.
>
> Better Automated Handling of Channel Additions/Deletions
> =======================================================
>
> Handling of channels additions and deletions in mythfilldatabase has
> always seemed way more complex to me than needed. Perhaps that
> because of the DataDirect complication (see above) or I'm just to
> naive to understand all of the subtleties. Either way, I hope to
> improve on it.
>
> Note that I'm primarily concerning myself with the
> mythfilldatabase/xmltv part of this problem for the time being. I
> know I can't totally ignore channel scanning for those tuner types
> that require it but major changes in the is not something I'm prepared
> to take on.
>
> Anyway, I'm looking to add new settings for each videosource to
> control channel additions and deletions. For additions, I'm thinking
> of a setting with values like "ignore", "add as never visible", "add
> as invisible", "add as visible" and "add as always visible". If it's
> possible to actually check the reception then also have an option for
> "add as detected". For deletions, I'm thinking of a setting with
> values like "ignore", "delete", "mark as invisible" and "mark as
> never visible".
>
> Better Automated Handling of Channel Updates
> ============================================
>
> Besides channels getting added and deleted, other channel information
> like names, callsigns and icons get updated over time. I don't know
> if mythfilldatabasse handles any of that or not. Gary Buhrmaster
> (surprise), though, has a script for doing much of this, at least for
> tv_grab_zz_sdjson_sqlite users. I'd like to see some of that
> functionality get rolled into MythTV proper. I don't know if that
> should be mythfilldatabase, mythutil or an entirely new utility at
> this point.
>
> One complicating factor is deciding how granularly to control the
> updates -- at the videosource level and/or the individual channel
> level. Also, which information to allow updates -- purely cosmetic
> stuff like name vs. callsign (with scheduling implications) vs. tuning
> stuff like frequency and multiplex. Gary's script allows very
> granular control which is great for me because I need it. Others
> probably don't and just makes things more complex for them.

Ooh, I forgot one other thing. I forgot it because it's big enough
feature that I gave it its own Google Keep note. It's also not
technically a channel managment feature but the inspiration for it
defintely came from here. This one is definitely in the "when I get
to it" category. I hope someone else with think it useful enought to
pick up the ball and run with it before then. Hint, hint, hint! :)

Add System Messages
===================

Thanks to Jean-Yves Avenard's work on notifications some years ago, we
have a way to inform users on all frontends of things that might need
immediate attention. That includes things like "tuning is taking too
long", a recording is failing" and "the backend is not responding".

There are lots of other things that users should be made aware of but
have to go look for on various other screens and sometimes even log
files. These include things like "channels were added or deleted"
(inspiration), "a scheduling conflict was created", "recordings were
missed or failed", "mythfilldatabase failed" and "a transcoding job
failed".

There should be some indication on the main menu screen and maybe even
all screens or just all menu screens when new messages are available.
In the messages area, users should be able to delete old messages and
mark new messages as read and also mark read messages as unread again.
In some cases, it would be nice to be able to go straight from the
message screen to another appropriate screen. For example, if
channels are added, have a link to go to the channel editor.

I've got some more notes for this but you get the idea. I'll save
them for when/if it's closer to becoming reality.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Mon, Jan 28, 2019 at 04:09:03PM +1300, Stephen Worthington wrote:
> On Sun, 27 Jan 2019 15:09:39 -0600, you wrote:
>
> [snip]
>
> An additional complication that you may not have thought about - when
> a channel changes its identity (station name, callsign, icon),
> obviously the recording rules for that callsign need to be updated
> also. But if you are using "power" recording rules, there will often
> be SQL stored in the recording rule that will contain a reference to
> the callsign and/or the chanid. It would be good if those rules could
> at least be flagged for manual fixing, even if an automated fix was
> not possible.

I had thought of it but didn't get put into my notes which is why I
forgot to bring it up. Thanks for reminding me. Yes, "this channel"
recording rules should be updated automatically. I don't it would be
wise to automatically update power rules too. Did you see my addendum
for system messages? One of the reasons for that is to inform the
user of such changes and indirectly remind them to take care of
anything that needs to be done manually.

> In this part of the world, channels are assigned XMLTV IDs by the EPG
> grabbing software that creates the xmltv data. Those names are
> normally made from the channel name found when scanning for channels.
> When the station identity changes, those names also need to be changed
> if you are trying to keep things consistent.
>
> Here are the current XMLTV IDs I am using for our free-to-air
> channels:
>
> tv1.freeviewnz.tv
> tv2.freeviewnz.tv
> tv3.freeviewnz.tv
> bravo.freeviewnz.tv
> maori-tv.freeviewnz.tv
> tv1-plus1.freeviewnz.tv
> tv2-plus1.freeviewnz.tv
> tv3-plus1.freeviewnz.tv
> bravo-plus1.freeviewnz.tv
> duke.freeviewnz.tv
> edge-tv.freeviewnz.tv
> prime.freeviewnz.tv
> choice.freeviewnz.tv
> hgtv.freeviewnz.tv
> parliament.freeviewnz.tv
> hope-channel.freeviewnz.tv
> aljazeera.freeviewnz.tv
> te-reo.freeviewnz.tv
> firstlight.freeviewnz.tv
> tvsn-shopping-channel.freeviewnz.tv
> chinese-tv8.freeviewnz.tv
> tv29.freeviewnz.tv
> tv33.freeviewnz.tv
> apna.freeviewnz.tv
> rnz-national.freeviewnz.tv
> rnz-concert.freeviewnz.tv
> basefm.freeviewnz.tv
> threelife.freeviewnz.tv
>
> and here are a few channels from our pay-TV satellite provider:
>
> cnbc.sky.co.nz
> cnn.sky.co.nz
> comedy-central.sky.co.nz
> concert-fm.radio.sky.co.nz
> country.music.sky.co.nz
> countrytv.sky.co.nz
> crime-investigation.sky.co.nz
>
> Another way I have thought about for keeping old channel data
> available is to simply create an "oldchannel" table. No longer
> available channels would be moved from "channel" to "oldchannel".
> Occasionally, housekeeping could be done to scan the recordings tables
> and check if there are any channels in "oldchannel" that no longer
> have matching recordings, and then delete those channels from
> "oldchannel". That could be done by an external script, as is done
> with find_orphans.py. Or it could be added to find_orphans.py. Using
> an oldchannel table greatly reduces the redundant data that might
> otherwise need to be stored in the "recorded" table and allows all of
> the old data for a channel to be kept. But it complicates the use of
> chanid values as they are now a primary key on two tables and need to
> be unique over both tables.

I'll consider recordedchannel (aka oldchannel) to be option 3. We
already have a recordedprogram and other recorded* tables for similar
uses. There are two issues I see.

First, as you noted, channels can change over time. Sometimes, they
change quite dramatically. I think Watch Recordings and Previously
Recorded should show the channel information as it was at the time of
recording. For example, I want my Hee Haw recordings to show as being
on The Nashville Network, not The National Network, not Spike TV and
not Paramount. We would need to add a time component, probably
starttime, to the table to prevent it from being changed by later
channel updates.

Second, the oldrecorded table is quite the different beast from the
recorded table. I don't see a new recordedchannel table being a good
fit for the oldrecorded table. oldrecorded has *a lot* more entries
and I don't see extra, time-accurate, channel information being as
important for oldrecorded as it is for recorded. Perhaps recorded and
oldrecorded should have different solutions. Option 3 for recorded
and option 1 for oldrecorded.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sun, Jan 27, 2019 at 9:09 PM David Engel <david@istwok.net> wrote:
> Please provide comments or other suggestions here.

I am in transit to another location, so when this
gets posted is unclear (next time wifi is close by).
Anyway, I do have a few thoughts (surprise), even
if I don't have time right at the moment to get
everything written down I would like to have
considered. So this is just some immediate ideas.

First, I philosophically do not like duplicate data
in tables. One table is the authoritative table, or
there are none (I don't have the ability to check
if that is a direct rule of Codd while I am writing
this, but it certainly is derivable from those rules).

The right answer for some of these issues is often
what are called effective dated tables (with the
additional of effective sequences for slightly more
complex update scenarios). That means every row
has a date it starts to be valid. So (using John's
example of free promos for a limited time period),
the row is currently marked with the last update
date with no visibility. There is an additional row
marked as being visible with the effective date of
(say) next Friday, and then there is another row
marked as invisible on the Monday after. At any
date one can determine visibility, as there is a
complete history and future for each row. To use
this table in many cases, one has to do what is
called a correlated sub-query, looking for the row
which is applicable for the date in question (often
on multiple different tables at the same time using
the appropriately dated rows for the query). This
means that the recordings on the promo channels
will be scheduled only during the visible period. It is
an elegant solution to such requirements. However,
it has a couple of substantial problems. First there
is the likelihood that probably only a very small
number of MythTV DB experts could write and
understand such queries. The second is that I am
not sure the current state of the mysql optimizer
performance on such selects and last I knew it
performed badly (as I understand it, it is a hard
problem in the general case (slightly over 20 years
ago, Oracle, which at the time was generally
considered to have one of the best query optimizers
on the planet, could end up performing full table
scans for certain correlated sub-queries for a
very popular ERP solution. Oracle fixed it when it
was identified, but it was apparently not something
trivial to get right, and the suggestion at the time
was that it was a targeted fix identifying the specific
type of correlated sub-query.)).

Since you are not likely going to effective date the
tables (you need more than just a few experts on
such things to be able to maintain such codes into
the future), I would expect you will need to copy
the data, and you should want 100% of the
information you will want to present as part of the
(old)recorded be copied. That would include at
least the channel name, number, channel icon, etc.
since (at least for cable) channel 123 may have
been ABC KABC today, but have been CBS KCBS
last week, and you want the data presented to be
valid for when the show was recorded.

btw, the icon data is differently problematic. It
is not (currently) identified by videosource, so
if one has multiple videosources, and the guide
data provides channel icon names that are the
same (perfectly possible, especially over time,
and has the same effective dated challenges),
you get collisions and incorrect over-writes.
There are a couple of options that have occured
to me (the easiest might be to prefix the icon
file names with the videosource number and a
date for uniqueness, but I think the more elegant
solution is an icon blob table indexed by a uuid,
which could be lazy trimmed when no references
from the (old)recorded and channel tables exist).
That might be a bridge (or db table) too far at
the moment.

That is all I have time for right now (hope this
makes it out to the ether at some point....)
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Mon, Jan 28, 2019 at 09:54:35AM +0000, Gary Buhrmaster wrote:
> On Sun, Jan 27, 2019 at 9:09 PM David Engel <david@istwok.net> wrote:
> > Please provide comments or other suggestions here.
>
> I am in transit to another location, so when this
> gets posted is unclear (next time wifi is close by).
> Anyway, I do have a few thoughts (surprise), even
> if I don't have time right at the moment to get
> everything written down I would like to have
> considered. So this is just some immediate ideas.
>
> First, I philosophically do not like duplicate data
> in tables. One table is the authoritative table, or
> there are none (I don't have the ability to check
> if that is a direct rule of Codd while I am writing
> this, but it certainly is derivable from those rules).
>
> The right answer for some of these issues is often
> what are called effective dated tables (with the
> additional of effective sequences for slightly more
> complex update scenarios). That means every row
> has a date it starts to be valid. So (using John's

I do just that with a database at work. However, I think that's
overkill for MythTV and "the perfect is the enemy of the good."
Still, I will let it percolate for a while as this is not the first
part I expect to work ong.

> example of free promos for a limited time period),
> the row is currently marked with the last update
> date with no visibility. There is an additional row
> marked as being visible with the effective date of
> (say) next Friday, and then there is another row
> marked as invisible on the Monday after. At any
> date one can determine visibility, as there is a
> complete history and future for each row. To use
> this table in many cases, one has to do what is
> called a correlated sub-query, looking for the row
> which is applicable for the date in question (often
> on multiple different tables at the same time using
> the appropriately dated rows for the query). This
> means that the recordings on the promo channels
> will be scheduled only during the visible period. It is
> an elegant solution to such requirements. However,
> it has a couple of substantial problems. First there
> is the likelihood that probably only a very small
> number of MythTV DB experts could write and
> understand such queries. The second is that I am
> not sure the current state of the mysql optimizer
> performance on such selects and last I knew it
> performed badly (as I understand it, it is a hard
> problem in the general case (slightly over 20 years
> ago, Oracle, which at the time was generally
> considered to have one of the best query optimizers
> on the planet, could end up performing full table
> scans for certain correlated sub-queries for a
> very popular ERP solution. Oracle fixed it when it
> was identified, but it was apparently not something
> trivial to get right, and the suggestion at the time
> was that it was a targeted fix identifying the specific
> type of correlated sub-query.)).
>
> Since you are not likely going to effective date the
> tables (you need more than just a few experts on
> such things to be able to maintain such codes into
> the future), I would expect you will need to copy
> the data, and you should want 100% of the
> information you will want to present as part of the
> (old)recorded be copied. That would include at
> least the channel name, number, channel icon, etc.
> since (at least for cable) channel 123 may have
> been ABC KABC today, but have been CBS KCBS
> last week, and you want the data presented to be
> valid for when the show was recorded.

For recorded, I'm almost completely in the option 3 camp now. That is
to have a recordedchannel table. We already have the precedents for
recordedprogram, et al and the code to populate and clean them up when
needed.

For oldrecorded, I'm still undecided. I personally would be fine with
saying that anything beyonhd channem number, name and maybe icon
simply aren't there for uses where oldrecorded is the primary table.

David

> btw, the icon data is differently problematic. It
> is not (currently) identified by videosource, so
> if one has multiple videosources, and the guide
> data provides channel icon names that are the
> same (perfectly possible, especially over time,
> and has the same effective dated challenges),
> you get collisions and incorrect over-writes.
> There are a couple of options that have occured
> to me (the easiest might be to prefix the icon
> file names with the videosource number and a
> date for uniqueness, but I think the more elegant
> solution is an icon blob table indexed by a uuid,
> which could be lazy trimmed when no references
> from the (old)recorded and channel tables exist).
> That might be a bridge (or db table) too far at
> the moment.
>
> That is all I have time for right now (hope this
> makes it out to the ether at some point....)
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
> Wiadomo?? napisana przez David Engel <david@istwok.net> w dniu 27.01.2019, o godz. 22:09:
>
> Here's a brief description of some of the things relating to channel
> management that I hope to work on. I'll likely start on some of them
> soon. I'll get to the others when/if I get to them. Please provide
> comments or other suggestions here.
>

David,

Maybe You will be interested in idea of automatic channel updates where DVB channels & muxes are auto detect added, moved and removed.

Attached patch is LVR work and maybe can be good starting point to include into channel management?

Patch description:

TV: Auto detect added, moved and removed DVB channels and muxes

When used in conjunction with the EIT scanner then most changes
in channel line-up can be automatically added to the database without
manual re-tuning.

In addition, the changes are sent to all frontends where they are
shown as on-screen notifications.

br
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Feb 02, 2019 at 11:07:16AM +0100, Piotr Oniszczuk wrote:
> > Wiadomo?? napisana przez David Engel <david@istwok.net> w dniu 27.01.2019, o godz. 22:09:
> >
> > Here's a brief description of some of the things relating to channel
> > management that I hope to work on. I'll likely start on some of them
> > soon. I'll get to the others when/if I get to them. Please provide
> > comments or other suggestions here.
> >
>
> David,
>
> Maybe You will be interested in idea of automatic channel updates where DVB channels & muxes are auto detect added, moved and removed.
>
> Attached patch is LVR work and maybe can be good starting point to include into channel management?
>
> Patch description:
>
> TV: Auto detect added, moved and removed DVB channels and muxes
>
> When used in conjunction with the EIT scanner then most changes
> in channel line-up can be automatically added to the database without
> manual re-tuning.

Piotr, for the near-term, I don't intend to address anything like
this. After tackling the things I've already identified, we'll see.

> In addition, the changes are sent to all frontends where they are
> shown as on-screen notifications.

That seems excessive. I think the messages feature I outlined would
be a better choice.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
[Quoting parts of my other emails for more context]

On Sun, Jan 27, 2019 at 03:09:39PM -0600, David Engel wrote:
> Remove Some Dependencies on the Channel Table
> =============================================
>
> When program entries are primarily read from the recorded table (Watch
> Recordings screen) and the oldrecorded table (Previously Recorded
> screen), they are augmented with data from the channel table. This is
> the reason that channel and videosource entries must be kept in the
> database even after they are no longer being used for new recordings.
> The intent of the following options is to remove these dependencies on
> the channel table so that channels and videosources can be deleted as
> soon as they are no longer used.
>
> Option 1. Add the relevant columns to the other tables. The recorded
> table would add channum, callsign and channame(*). The oldrecorded
> table would add channum and channame.
>
> Option 1a. The same as option 1 but add the new columns to the
> recordedprogram table instead of the recorded table. I prefer plain,
> option 1 for simplicity and keeping the program and recordedprogram
> tables with the same columns.
>
> Option 2. Add some new indication in the channel table for "Deleted"
> channels. This could work with the cost being more complexity to
> ignore deleted channels in the appropriate places and deciding when to
> actually delete an entry or mark it as deleted.
>
> Right now, I prefer option 1 due to its simplicity. One downside is
> that other potentially useful data about the channel would be lost
> when it's deleted. For example, what if a themer wanted to show the
> channel icon in Watch Recordings instead of or in addition to the
> preview image?
>
> (*)A couple of other columns from the channel table are pulled in here
> but I believe they are only relevant immediately after the program was
> recorded. Returning NULLs if the channel is deleted much later should
> not cause a problem.

On Sun, Jan 27, 2019 at 11:33:45PM -0600, David Engel wrote:
> I'll consider recordedchannel (aka oldchannel) to be option 3. We
> already have a recordedprogram and other recorded* tables for similar
> uses. There are two issues I see.

On Mon, Jan 28, 2019 at 08:25:40PM -0600, David Engel wrote:
> On Mon, Jan 28, 2019 at 09:54:35AM +0000, Gary Buhrmaster wrote:
> > First, I philosophically do not like duplicate data
> > in tables. One table is the authoritative table, or
> > there are none (I don't have the ability to check
> > if that is a direct rule of Codd while I am writing
> > this, but it certainly is derivable from those rules).
> >
> > The right answer for some of these issues is often
> > what are called effective dated tables (with the
> > additional of effective sequences for slightly more
> > complex update scenarios). That means every row
> > has a date it starts to be valid. So (using John's
>
> I do just that with a database at work. However, I think that's
> overkill for MythTV and "the perfect is the enemy of the good."
> Still, I will let it percolate for a while as this is not the first
> part I expect to work ong.

> > Since you are not likely going to effective date the
> > tables (you need more than just a few experts on
> > such things to be able to maintain such codes into
> > the future), I would expect you will need to copy
> > the data, and you should want 100% of the
> > information you will want to present as part of the
> > (old)recorded be copied. That would include at
> > least the channel name, number, channel icon, etc.
> > since (at least for cable) channel 123 may have
> > been ABC KABC today, but have been CBS KCBS
> > last week, and you want the data presented to be
> > valid for when the show was recorded.
>
> For recorded, I'm almost completely in the option 3 camp now. That is
> to have a recordedchannel table. We already have the precedents for
> recordedprogram, et al and the code to populate and clean them up when
> needed.
>
> For oldrecorded, I'm still undecided. I personally would be fine with
> saying that anything beyonhd channem number, name and maybe icon
> simply aren't there for uses where oldrecorded is the primary table.

I have about 90% of an option 3 coded and tested. It creates new,
recordedchannel (for recordedprogram) and oldchannel (for oldrecorded)
tables. The last remaining part has to do with icon handling.
Becuase the icon path is not part of the Program/RecordingInfo object,
it is fetched on demand using ChannelUtil::GetIcon() which blindly
uses the channel table. It needs to be modified to conditionally use
channel, recordedchannel or oldchannel as appropriate.

I'm about to throw it all out in favor of a new, option 2a.

The new option adds another, new value to channel.visible for
"deleted". Deleted channels are like existing, invisible channels
with a a few, new exceptions. They aren't shown in the channel
editor, aren't popluated with guide data and are deleted from the
channel table when they are no longer referenced by the
recordedprogram and oldrecorded tables.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Fri, Apr 12, 2019 at 8:24 PM David Engel <david@istwok.net> wrote:

> I have about 90% of an option 3 coded and tested. It creates new,
> recordedchannel (for recordedprogram) and oldchannel (for oldrecorded)
> tables. The last remaining part has to do with icon handling.
> Becuase the icon path is not part of the Program/RecordingInfo object,
> it is fetched on demand using ChannelUtil::GetIcon() which blindly
> uses the channel table. It needs to be modified to conditionally use
> channel, recordedchannel or oldchannel as appropriate.

One of the problems, which I believe you are aware of,
with icons is that (at least with US cable and satellite)
that the proper icon (and the callsign and name) for
channel 123 may be for the USA network on 2017/04,
and be the TNT network in 2019/03 (made up networks
and dates, of course). So if the icon is important, you
have to copy the info at the time of record. Even if
the videosource still exists, and is still being used,
the actual "station" on the "channel" may now be
different.

Without doing much evaluation, I think the only way
that a "deleted" flag makes a bit of sense is that if
every update to a channel makes the current channel
"deleted", replaced whatever is the new values may
be and (of course) a new chanid, which may have
implications of existing guide data.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
About channel icons. I was thinking along the following lines:
- Use a separate database table for the icons, with fields "callsign"
and/or "name" (channel name) and "icon" (icon path or maybe the icon
as a blob inside the database).
- Remove field "icon" (icon path) from channel.
- Access the icon via lookup on callsign and/or name.
This would give the icons a life independent of the channels and, more
important, independent of the chanid. Which means it is possible to
delete all channels, do a new scan, and still have the icons. This
would also reduce the need for users to write SQL scripts to
repopulate the "icon" fields in the channel table after a new scan. To
work with table oldrecorded the channel callsign and/or name is needed
but that was the plan anyway.

Groetjes,
Klaas.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Apr 13, 2019 at 4:13 PM Klaas de Waal <klaas.de.waal@gmail.com> wrote:

> - Access the icon via lookup on callsign and/or name.

Unfortunately, again, in the US, the callsign and/or name
is not always sufficient for lookup (there are different logos
for the SD and HD formats in some cases). And the logo
can (and has) change over the years (which may, or may
not be, an issue, but when a local changed from an NBC
affiliate to independent, the NBC part disappeared in the
logo, for example). Oh, and furthermore, the same callsign
and/or name can be used on two different videosources in
different ways, so you also have to include the videosource
id (which cannot be allowed to be reused to avoid future
conflicts if you use it as a discriminator).

And I have little idea how other parts of the world
handle these cases.

All of these may end up being things David decides
are enemies of the good enough, but they all do exist
in real world MythTV databases (and I have been
running these codes long enough to have run into
all of them).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Apr 13, 2019 at 12:39:44AM +0000, Gary Buhrmaster wrote:
> On Fri, Apr 12, 2019 at 8:24 PM David Engel <david@istwok.net> wrote:
>
> > I have about 90% of an option 3 coded and tested. It creates new,
> > recordedchannel (for recordedprogram) and oldchannel (for oldrecorded)
> > tables. The last remaining part has to do with icon handling.
> > Becuase the icon path is not part of the Program/RecordingInfo object,
> > it is fetched on demand using ChannelUtil::GetIcon() which blindly
> > uses the channel table. It needs to be modified to conditionally use
> > channel, recordedchannel or oldchannel as appropriate.
>
> One of the problems, which I believe you are aware of,
> with icons is that (at least with US cable and satellite)
> that the proper icon (and the callsign and name) for
> channel 123 may be for the USA network on 2017/04,
> and be the TNT network in 2019/03 (made up networks
> and dates, of course). So if the icon is important, you
> have to copy the info at the time of record. Even if
> the videosource still exists, and is still being used,
> the actual "station" on the "channel" may now be
> different.
>
> Without doing much evaluation, I think the only way
> that a "deleted" flag makes a bit of sense is that if
> every update to a channel makes the current channel
> "deleted", replaced whatever is the new values may
> be and (of course) a new chanid, which may have
> implications of existing guide data.

I'm very aware of the issues. I didn't want to get bogged down in the
minutia of changed channels versus new channels. Ultimately, that's
the user's decision. But since you ask, I'll give my personal opinion
with a few examples.

When the callsign changes from TNN to SPIKE to PAR, those are new
channels. The existing channel should be marked as deleted and a new
channel with a new chanid should be added. Old recording will
continue to reference the old channel as long as the exist and new
recordings will reference the new channel.

When the Cable Co. moves ESPN from channel 123 to channel 456, that's
not a new channel. The existing channel is update and life goes on.
It won't bother me in the least that recordings really done on channel
123 might now show as done on channel 456.

When Outdoor Life Network changes its name to OLN or Science Fiction
changes to SYFY and neither substantially changes their programming,
those probably aren't new channel either. Someone else might feel
differently and they're free to do a delete old/add new if they like.

I very much like my new proposal for keeping the simplicity of a
single table with a single column controlling visibility features
and also has the ability to preserve historical entries when they
are desired.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Apr 13, 2019 at 4:47 PM David Engel <david@istwok.net> wrote:
>
> or Science Fiction
> changes to SYFY and neither substantially changes their programming,
> those probably aren't new channel either.

Although when Sci Fi changed their name to SyFy, they
DID substantially change their programming (but that
is a different tirade regarding their programming pivot).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Apr 13, 2019 at 4:42 PM Gary Buhrmaster
<gary.buhrmaster@gmail.com> wrote:
>
> On Sat, Apr 13, 2019 at 4:13 PM Klaas de Waal <klaas.de.waal@gmail.com> wrote:
>
> > - Access the icon via lookup on callsign and/or name.
>

And so one does not think I am just pointing
to issues, here is what I think would work
(with zero actual implementation) if one
wanted to keep things "mostly" complete,
but with a LOT more work in the code to
implement (and there is also, as I understand
it, a project concern on not doing things
like using database blobs, so I guess one
could do the same thing by using the file
system lookups if one wanted, but additional
safeguards regarding file system unavailability
would need to be added).

New DB table (ICONBLOB), with column
ICONUUID (unique index) a column ICON
which is a blob of the icon itself.

Anytime a new logo is downloaded or
assigned to a channel, it gets a new
ICONBLOB row (random ICONUUID
assigned) and the channel is assigned
the new ICONUUID. So every channel
has the current ICONUUID.

(old?)Recorded point to the ICONUUID
at the time of the recording.

Any lookups/retrievals would use the
ICONUUID value into the ICONBLOB
table to retrieve the icon itself.

A background db cleanup of the form:

delete from ICONBLOB where UUID
not in (select ICONUUID from channel
UNION
select ICONUUID from recorded)




Yes, it is (probably? absolutely?) overkill.
But it has a certain elegance that is
appealing in the abstract. Especially
for someone not actually doing the
code (which I am not) It is always easy
spending OPM and OPT.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Sat, Apr 13, 2019 at 04:53:57PM +0000, Gary Buhrmaster wrote:
> On Sat, Apr 13, 2019 at 4:47 PM David Engel <david@istwok.net> wrote:
> >
> > or Science Fiction
> > changes to SYFY and neither substantially changes their programming,
> > those probably aren't new channel either.
>
> Although when Sci Fi changed their name to SyFy, they
> DID substantially change their programming (but that
> is a different tirade regarding their programming pivot).

I hadn't realized they changed that much. After the very few SciFi
shows I watched at the time either ended or got cancelled, nothing
else appealed to me. I really haven't watched the network since
except for the occasional movie.

Anyway, what I've proposed can handle it. When I get to the actual
handling of the changed channels part (which likely won't be for a
while), we can support flexible policies for what changes among
callsign, name, number, icon or whatever trigger a delete/add instead
of an update.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On 12/04/2019 21:24, David Engel wrote:
> [Quoting parts of my other emails for more context]
>
> On Sun, Jan 27, 2019 at 03:09:39PM -0600, David Engel wrote:
>> Remove Some Dependencies on the Channel Table
>> =============================================
>>
>> When program entries are primarily read from the recorded table (Watch
>> Recordings screen) and the oldrecorded table (Previously Recorded
>> screen), they are augmented with data from the channel table. This is
>> the reason that channel and videosource entries must be kept in the
>> database even after they are no longer being used for new recordings.
>> The intent of the following options is to remove these dependencies on
>> the channel table so that channels and videosources can be deleted as
>> soon as they are no longer used.
>>
>> Option 1. Add the relevant columns to the other tables. The recorded
>> table would add channum, callsign and channame(*). The oldrecorded
>> table would add channum and channame.
>>
>> Option 1a. The same as option 1 but add the new columns to the
>> recordedprogram table instead of the recorded table. I prefer plain,
>> option 1 for simplicity and keeping the program and recordedprogram
>> tables with the same columns.
>>
>> Option 2. Add some new indication in the channel table for "Deleted"
>> channels. This could work with the cost being more complexity to
>> ignore deleted channels in the appropriate places and deciding when to
>> actually delete an entry or mark it as deleted.
>>
>> Right now, I prefer option 1 due to its simplicity. One downside is
>> that other potentially useful data about the channel would be lost
>> when it's deleted. For example, what if a themer wanted to show the
>> channel icon in Watch Recordings instead of or in addition to the
>> preview image?
>>
>> (*)A couple of other columns from the channel table are pulled in here
>> but I believe they are only relevant immediately after the program was
>> recorded. Returning NULLs if the channel is deleted much later should
>> not cause a problem.
>
> On Sun, Jan 27, 2019 at 11:33:45PM -0600, David Engel wrote:
>> I'll consider recordedchannel (aka oldchannel) to be option 3. We
>> already have a recordedprogram and other recorded* tables for similar
>> uses. There are two issues I see.
>
> On Mon, Jan 28, 2019 at 08:25:40PM -0600, David Engel wrote:
>> On Mon, Jan 28, 2019 at 09:54:35AM +0000, Gary Buhrmaster wrote:
>>> First, I philosophically do not like duplicate data
>>> in tables. One table is the authoritative table, or
>>> there are none (I don't have the ability to check
>>> if that is a direct rule of Codd while I am writing
>>> this, but it certainly is derivable from those rules).
>>>
>>> The right answer for some of these issues is often
>>> what are called effective dated tables (with the
>>> additional of effective sequences for slightly more
>>> complex update scenarios). That means every row
>>> has a date it starts to be valid. So (using John's
>>
>> I do just that with a database at work. However, I think that's
>> overkill for MythTV and "the perfect is the enemy of the good."
>> Still, I will let it percolate for a while as this is not the first
>> part I expect to work ong.
>
>>> Since you are not likely going to effective date the
>>> tables (you need more than just a few experts on
>>> such things to be able to maintain such codes into
>>> the future), I would expect you will need to copy
>>> the data, and you should want 100% of the
>>> information you will want to present as part of the
>>> (old)recorded be copied. That would include at
>>> least the channel name, number, channel icon, etc.
>>> since (at least for cable) channel 123 may have
>>> been ABC KABC today, but have been CBS KCBS
>>> last week, and you want the data presented to be
>>> valid for when the show was recorded.
>>
>> For recorded, I'm almost completely in the option 3 camp now. That is
>> to have a recordedchannel table. We already have the precedents for
>> recordedprogram, et al and the code to populate and clean them up when
>> needed.
>>
>> For oldrecorded, I'm still undecided. I personally would be fine with
>> saying that anything beyonhd channem number, name and maybe icon
>> simply aren't there for uses where oldrecorded is the primary table.
>
> I have about 90% of an option 3 coded and tested. It creates new,
> recordedchannel (for recordedprogram) and oldchannel (for oldrecorded)
> tables. The last remaining part has to do with icon handling.
> Becuase the icon path is not part of the Program/RecordingInfo object,
> it is fetched on demand using ChannelUtil::GetIcon() which blindly
> uses the channel table. It needs to be modified to conditionally use
> channel, recordedchannel or oldchannel as appropriate.
>

I actually like the sound of this.

Icon's are a bit problematic these days, since lyngsat implemented
linking protection, myth can't automatically download them using the
built in icon downloader.

> I'm about to throw it all out in favor of a new, option 2a.
>
> The new option adds another, new value to channel.visible for
> "deleted". Deleted channels are like existing, invisible channels
> with a a few, new exceptions. They aren't shown in the channel
> editor, aren't popluated with guide data and are deleted from the
> channel table when they are no longer referenced by the
> recordedprogram and oldrecorded tables.
>

Maybe i update my channels in a funny way ;-)
truncate channel;
truncate eit_cache;
truncate program;
rescan
restart backend


Regards
Stuart
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Tue, Apr 16, 2019 at 01:44:20PM +0100, Stuart Auchterlonie wrote:
> On 12/04/2019 21:24, David Engel wrote:
> > [Quoting parts of my other emails for more context]
> >
> > On Sun, Jan 27, 2019 at 03:09:39PM -0600, David Engel wrote:
> >> Remove Some Dependencies on the Channel Table
> >> =============================================
> >>
> >> When program entries are primarily read from the recorded table (Watch
> >> Recordings screen) and the oldrecorded table (Previously Recorded
> >> screen), they are augmented with data from the channel table. This is
> >> the reason that channel and videosource entries must be kept in the
> >> database even after they are no longer being used for new recordings.
> >> The intent of the following options is to remove these dependencies on
> >> the channel table so that channels and videosources can be deleted as
> >> soon as they are no longer used.
> >>
> >> Option 1. Add the relevant columns to the other tables. The recorded
> >> table would add channum, callsign and channame(*). The oldrecorded
> >> table would add channum and channame.
> >>
> >> Option 1a. The same as option 1 but add the new columns to the
> >> recordedprogram table instead of the recorded table. I prefer plain,
> >> option 1 for simplicity and keeping the program and recordedprogram
> >> tables with the same columns.
> >>
> >> Option 2. Add some new indication in the channel table for "Deleted"
> >> channels. This could work with the cost being more complexity to
> >> ignore deleted channels in the appropriate places and deciding when to
> >> actually delete an entry or mark it as deleted.
> >>
> >> Right now, I prefer option 1 due to its simplicity. One downside is
> >> that other potentially useful data about the channel would be lost
> >> when it's deleted. For example, what if a themer wanted to show the
> >> channel icon in Watch Recordings instead of or in addition to the
> >> preview image?
> >>
> >> (*)A couple of other columns from the channel table are pulled in here
> >> but I believe they are only relevant immediately after the program was
> >> recorded. Returning NULLs if the channel is deleted much later should
> >> not cause a problem.
> >
> > On Sun, Jan 27, 2019 at 11:33:45PM -0600, David Engel wrote:
> >> I'll consider recordedchannel (aka oldchannel) to be option 3. We
> >> already have a recordedprogram and other recorded* tables for similar
> >> uses. There are two issues I see.
> >
> > On Mon, Jan 28, 2019 at 08:25:40PM -0600, David Engel wrote:
> >> On Mon, Jan 28, 2019 at 09:54:35AM +0000, Gary Buhrmaster wrote:
> >>> First, I philosophically do not like duplicate data
> >>> in tables. One table is the authoritative table, or
> >>> there are none (I don't have the ability to check
> >>> if that is a direct rule of Codd while I am writing
> >>> this, but it certainly is derivable from those rules).
> >>>
> >>> The right answer for some of these issues is often
> >>> what are called effective dated tables (with the
> >>> additional of effective sequences for slightly more
> >>> complex update scenarios). That means every row
> >>> has a date it starts to be valid. So (using John's
> >>
> >> I do just that with a database at work. However, I think that's
> >> overkill for MythTV and "the perfect is the enemy of the good."
> >> Still, I will let it percolate for a while as this is not the first
> >> part I expect to work ong.
> >
> >>> Since you are not likely going to effective date the
> >>> tables (you need more than just a few experts on
> >>> such things to be able to maintain such codes into
> >>> the future), I would expect you will need to copy
> >>> the data, and you should want 100% of the
> >>> information you will want to present as part of the
> >>> (old)recorded be copied. That would include at
> >>> least the channel name, number, channel icon, etc.
> >>> since (at least for cable) channel 123 may have
> >>> been ABC KABC today, but have been CBS KCBS
> >>> last week, and you want the data presented to be
> >>> valid for when the show was recorded.
> >>
> >> For recorded, I'm almost completely in the option 3 camp now. That is
> >> to have a recordedchannel table. We already have the precedents for
> >> recordedprogram, et al and the code to populate and clean them up when
> >> needed.
> >>
> >> For oldrecorded, I'm still undecided. I personally would be fine with
> >> saying that anything beyonhd channem number, name and maybe icon
> >> simply aren't there for uses where oldrecorded is the primary table.
> >
> > I have about 90% of an option 3 coded and tested. It creates new,
> > recordedchannel (for recordedprogram) and oldchannel (for oldrecorded)
> > tables. The last remaining part has to do with icon handling.
> > Becuase the icon path is not part of the Program/RecordingInfo object,
> > it is fetched on demand using ChannelUtil::GetIcon() which blindly
> > uses the channel table. It needs to be modified to conditionally use
> > channel, recordedchannel or oldchannel as appropriate.
> >
>
> I actually like the sound of this.
>
> Icon's are a bit problematic these days, since lyngsat implemented
> linking protection, myth can't automatically download them using the
> built in icon downloader.
>
> > I'm about to throw it all out in favor of a new, option 2a.
> >
> > The new option adds another, new value to channel.visible for
> > "deleted". Deleted channels are like existing, invisible channels
> > with a a few, new exceptions. They aren't shown in the channel
> > editor, aren't popluated with guide data and are deleted from the
> > channel table when they are no longer referenced by the
> > recordedprogram and oldrecorded tables.
> >
>
> Maybe i update my channels in a funny way ;-)
> truncate channel;
> truncate eit_cache;
> truncate program;
> rescan
> restart backend

I'd consider that funny, and highly trusting that rescanning will
re-use the same chanids. With what I've proposed, you wouldn't need
to change much and you also wouldn't need to rely on getting the same
chanids. Instead of "truncate channel", you'd run "update channel set
visible=<value-for-deleted>".

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On Tue, Apr 16, 2019 at 12:44 PM Stuart Auchterlonie
<stuarta@squashedfrog.net> wrote:

> Icon's are a bit problematic these days, since lyngsat implemented
> linking protection, myth can't automatically download them using the
> built in icon downloader.

Some xmltv grabbers have reasonable channel
icons available (of course some do not), but
EIT only obviously has no automated equivalent.

Since have been using sources that have
upstream curated icons for over a decade,
I tend to forget about the often time consuming
work that I performed doing the icon research
and selection manually when that was
required.

> Maybe i update my channels in a funny way ;-)
> truncate channel;
> truncate eit_cache;
> truncate program;
> rescan
> restart backend

If you insist on doing it different, at
least also truncate programgenres,
programrating, and credits to avoid
opportunities (for all sorts of artifacts).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Channel Management Ideas and Plans [ In reply to ]
On 16/04/2019 16:48, David Engel wrote:
> On Tue, Apr 16, 2019 at 01:44:20PM +0100, Stuart Auchterlonie wrote:
>>
>> Maybe i update my channels in a funny way ;-)
>> truncate channel;
>> truncate eit_cache;
>> truncate program;
>> rescan
>> restart backend
>
> I'd consider that funny, and highly trusting that rescanning will
> re-use the same chanids. With what I've proposed, you wouldn't need
> to change much and you also wouldn't need to rely on getting the same
> chanids. Instead of "truncate channel", you'd run "update channel set
> visible=<value-for-deleted>".
>
> David
>

Yes they do get the same chanid's, and when they don't it's because
the channel numbers have changed (and / or the channel lineup)
which is the normal reason to rescan here.


Regards
Stuart
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org