Mailing List Archive

Guide service API "Group By" usage
The Guide API uses "Group by Callsign" with the result that if you have
two channels with the same callsign, one gets dropped off the list. Is
there any reason for this? I plan to fix it with the other bugs I am
fixing, if nobody knows of a reason for grouping that way.

Comcast have a setup where many channels appears at least twice in HD
and once in SD. The two HD versions have the same callsign. The second
HD version is in the 1000+ range and grouped in ranges in some way they
think is helpful for channel surfing.

Peter

-------- Forwarded Message --------
Subject: Re: Ticket #13589: Bugs in backend serviceAPIs
Date: Fri, 25 Sep 2020 18:48:23 -0000
From: MythTV <noreply@mythtv.org>
Reply-To: mythtv-dev@mythtv.org
CC: mythtv-commits@mythtv.org



#13589: Bugs in backend serviceAPIs
---------------------------------------------+-----------------------------
Reporter: Peter Bennett | Owner: Peter
| Bennett
Type: Bug Report - General | Status: assigned
Priority: minor | Milestone:
| needs_triage
Component: MythTV - Services API - Backend | Version: Master Head
Severity: medium | Resolution:
Keywords: | Ticket locked: 0
---------------------------------------------+-----------------------------

Comment (by Peter Bennett):

The last bug listed, !Guide/GetProgramGuide, uses "Group By !CallSign",
with the result that if two or more channels have the same call sign, only
one of them is included in the list. With Comcast, the same station is
broadcast on several channels, and if you have your channels set up like
that you likely want to see the listings the same way. I am changing this
to no longer group that way.

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13589#comment:8>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Guide service API "Group By" usage [ In reply to ]
On Fri, Sep 25, 2020 at 6:57 PM Peter Bennett <pb.mythtv@gmail.com> wrote:
>
> The second HD version is in the 1000+ range and grouped in ranges in some way they think is helpful for channel surfing.
>

This (re) grouping also addressed some litigation
(a new entrant made a complaint they were being
disenfranchised because they were not near the
other channels of their genre for those that still
primarily using their up/down buttons to navigate
their hundreds of channels (I suppose pushing
buttons may have been more exercise than those
people would otherwise get, so there is that)).

And while I personally prefer the MCLU (and
only the MCLU(*)) layout primarily because it
makes scanning the channel lineups easier to
prepare my regular(**) guide service provider
correction requests, if I want a particular channel
for live TV I tune it explicitly, even if I suppose I
could hit "up/up/up/up/down" (the last two were
the all too often "whoops, went too far"
correction mode during the surf) to get there.






(*) Until Comcast decided to change the rules
about the MCLU always being the best available
channel (HD or SD, as available) as they move
to IPTV and leave linear QAM (and the goals of
the MCLU) behind. Now I have to do a more
detailed review of the entire mapping (as an
explanation, as the MCLU covered all the
channels, a request to change an MCLU channel
resulted in Comcast also updating the ancillary
SD and HD channels in their guide data, which
meant I only needed to check one place; now
I have to check three places (which is even
more than the two pre-MCLU places to check)).

(**) I am a bit behind in that review, as Comcast
has not added any of the 3xxx IPTV channels
to their guide data feed either (and while MythTV
cannot access those channels, it should be
in the guide data for presentation for other
consumers).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Guide service API "Group By" usage [ In reply to ]
On 9/25/20 2:56 PM, Peter Bennett wrote:
>
> The Guide API uses "Group by Callsign" with the result that if you
> have two channels with the same callsign, one gets dropped off the
> list. Is there any reason for this? I plan to fix it with the other
> bugs I am fixing, if nobody knows of a reason for grouping that way.
>
> Comcast have a setup where many channels appears at least twice in HD
> and once in SD. The two HD versions have the same callsign. The second
> HD version is in the 1000+ range and grouped in ranges in some way
> they think is helpful for channel surfing.
>
> Peter
>
>
On reconsidering this, I realized that if you have more than 1 "Input
Connection" you could have the same channels repeated, for example on
cable and over-the-air, and in that case will not want your channels
duplicated. So I think I should rather leave this as is.

Peter

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Guide service API "Group By" usage [ In reply to ]
On 09/27/2020 12:58 PM, Peter Bennett wrote:
>
> On 9/25/20 2:56 PM, Peter Bennett wrote:
>>
>> The Guide API uses "Group by Callsign" with the result that if you
>> have two channels with the same callsign, one gets dropped off the
>> list. Is there any reason for this? I plan to fix it with the other
>> bugs I am fixing, if nobody knows of a reason for grouping that way.
>>
>> Comcast have a setup where many channels appears at least twice in HD
>> and once in SD. The two HD versions have the same callsign. The
>> second HD version is in the 1000+ range and grouped in ranges in some
>> way they think is helpful for channel surfing.
>>
>> Peter
>>
>>
> On reconsidering this, I realized that if you have more than 1 "Input
> Connection" you could have the same channels repeated, for example on
> cable and over-the-air, and in that case will not want your channels
> duplicated. So I think I should rather leave this as is.

Also, in MythTV, the call sign has been used to indicate a channel's
uniqueness. Two channels with the same call sign are treated as the
same for the purposes of scheduling (therefore, no 2 channels should be
given the same call sign unless the content they broadcast is
substantially identical).

The channel number is used to indicate that channels are to be treated
the same for Live TV . If you tune a channel in Live TV by typing
channel number 13, you don't care which channel number 13 it
tunes--regardless of source or input. If you have a preference which is
tuned in Live TV, the channels should be given different channel numbers
to allow differentiation when tuning Live TV by channel number.

Both call sign and channel number are user-editable so that users can
change either or both of them. So a user can have channels using WGBH
and WGBH-HD for call signs to differentiate a standard definition and
high definition version (or whatever indicator the user chooses) for the
purposes of recording scheduling with the "this channel" filter. Or a
user could have a channel 13 for the channel the user prefers to tune
in Live TV and channel 1013 for one that's less desirable--for example
if 13 is usable from multiple inputs but 1013 is only usable from a
single input using, for example, a cable STB.

In theory, the guide should only condense multiple channels to a single
listing if both the call sign and channel number are identical for the
channels. If you're saying that's not the case in the services API or
something (it condenses if the call sign is identical regardless of the
channel number), that may be a bug.

Mike
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Guide service API "Group By" usage [ In reply to ]
On 9/28/20 9:25 AM, Michael T. Dean wrote:
> On 09/27/2020 12:58 PM, Peter Bennett wrote:
>>
>> On 9/25/20 2:56 PM, Peter Bennett wrote:
>>>
>>> The Guide API uses "Group by Callsign" with the result that if you
>>> have two channels with the same callsign, one gets dropped off the
>>> list. Is there any reason for this? I plan to fix it with the other
>>> bugs I am fixing, if nobody knows of a reason for grouping that way.
>>>
>>> Comcast have a setup where many channels appears at least twice in
>>> HD and once in SD. The two HD versions have the same callsign. The
>>> second HD version is in the 1000+ range and grouped in ranges in
>>> some way they think is helpful for channel surfing.
>>>
>>> Peter
>>>
>>>
>> On reconsidering this, I realized that if you have more than 1 "Input
>> Connection" you could have the same channels repeated, for example on
>> cable and over-the-air, and in that case will not want your channels
>> duplicated. So I think I should rather leave this as is.
>
> Also, in MythTV, the call sign has been used to indicate a channel's
> uniqueness.  Two channels with the same call sign are treated as the
> same for the purposes of scheduling (therefore, no 2 channels should
> be given the same call sign unless the content they broadcast is
> substantially identical).
>
> The channel number is used to indicate that channels are to be treated
> the same for Live TV .  If you tune a channel in Live TV by typing
> channel number 13, you don't care which channel number 13 it
> tunes--regardless of source or input.  If you have a preference which
> is tuned in Live TV, the channels should be given different channel
> numbers to allow differentiation when tuning Live TV by channel number.
>
> Both call sign and channel number are user-editable so that users can
> change either or both of them.  So a user can have channels using WGBH
> and WGBH-HD for call signs to differentiate a standard definition and
> high definition version (or whatever indicator the user chooses) for
> the purposes of recording scheduling with the "this channel" filter. 
> Or a user could have a  channel 13 for the channel the user prefers to
> tune in Live TV and channel 1013 for one that's less desirable--for
> example if 13 is usable from multiple inputs but 1013 is only usable
> from a single input using, for example, a cable STB.
>
> In theory, the guide should only condense multiple channels to a
> single listing if both the call sign and channel number are identical
> for the channels.  If you're saying that's not the case in the
> services API or something (it condenses if the call sign is identical
> regardless of the channel number), that may be a bug.
>
> Mike
>
Yes that is what the services API does for the GetProgramGuide (Group by
call sign and not channel number). My particular case was where two
channels have the same call sign but different numbers and only one showed.

Here in the USA with cable we prefer channel number, the callsigns
filled in by the guide service are not consistent. Many times they are
some sort of abbreviation of the channel name, sometimes with -DT, -DT2,
-DT3 etc. appended.

I recently came across somebody (I don't know where located) who has
null channel numbers. I did not think that was allowed but it may mess
up grouping by channel number. I am not sure if SQL will correctly group
items that have null in their group by column.

Should null channel numbers be allowed?

Peter

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Guide service API "Group By" usage [ In reply to ]
On 09/28/2020 11:43 AM, Peter Bennett wrote:
> On 9/28/20 9:25 AM, Michael T. Dean wrote:
>>> On 9/25/20 2:56 PM, Peter Bennett wrote:
>>>>
>>>> The Guide API uses "Group by Callsign" with the result that if you
>>>> have two channels with the same callsign, one gets dropped off the
>>>> list. Is there any reason for this? I plan to fix it with the other
>>>> bugs I am fixing, if nobody knows of a reason for grouping that way.
>>>>
...
...
>> In theory, the guide should only condense multiple channels to a
>> single listing if both the call sign and channel number are identical
>> for the channels. If you're saying that's not the case in the
>> services API or something (it condenses if the call sign is identical
>> regardless of the channel number), that may be a bug.
>>
> Yes that is what the services API does for the GetProgramGuide (Group
> by call sign and not channel number). My particular case was where two
> channels have the same call sign but different numbers and only one
> showed.

Yeah, TTBOMK, we still show multiple rows if either call sign or channel
number differs in the frontend guide and should in all other guides
(MythWeb and/or backend services). I don't have duplicate channels, so
it's possible things have been changed in recent years and I didn't
notice, but it seems to be a bug to me. With the distinctions between
identical channel numbers, identical call signs, and identical channel
numbers and call signs I described in my previous message, we seem to
cover cases that allow users fine-grained control over "what kind of
duplicate channel is this," so I wouldn't expect we should remove some
portion of that control. I'd think the service should be changed to
condense channels like the frontend channel guide.

> Here in the USA with cable we prefer channel number, the callsigns
> filled in by the guide service are not consistent. Many times they are
> some sort of abbreviation of the channel name, sometimes with -DT,
> -DT2, -DT3 etc. appended.
>
> I recently came across somebody (I don't know where located) who has
> null channel numbers. I did not think that was allowed but it may mess
> up grouping by channel number.

Yeah, though it shouldn't have an effect on scheduling, it would almost
definitely have negative effects elsewhere (such as Live TV's inability
to tune by channel number, display problems, or in transmitting
ProgramInfo using Myth backend protocol--because of null-terminated
strings).

> I am not sure if SQL will correctly group items that have null in
> their group by column.
>
> Should null channel numbers be allowed?

The column is defined as NOT NULL:

https://github.com/MythTV/mythtv/blob/master/mythtv/libs/libmythtv/dbcheck.cpp#L3894

and I don't see any changes anywhere that removed that restriction (nor
can I imagine changes that would do that). Not sure what the preferred
approach is regarding coding to handle that invalid data transparently
or requiring valid data, but we probably should at least alert the user
that their DB (and, seemingly DB schema) is borked if we notice
that--whether we fail hard or "just try and see" or insert some value in
code when reading null from the DB column to try to prevent errors. My
personal preference is to not spend a lot of time coding to handle
invalid data, but to log a message that makes it possible for the user
to figure out what's wrong and correct it. That said, I'm not at all
against bulletproof (or bullet-resistant?) code. (So, I guess that's
completely useless advice, and maybe someone else can be more helpful. :)

Mike
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Guide service API "Group By" usage [ In reply to ]
On 9/28/20 1:28 PM, Michael T. Dean wrote:
>
>> I am not sure if SQL will correctly group items that have null in
>> their group by column.
>>
>> Should null channel numbers be allowed?
>
> The column is defined as NOT NULL:
>
> https://github.com/MythTV/mythtv/blob/master/mythtv/libs/libmythtv/dbcheck.cpp#L3894
>
>
> and I don't see any changes anywhere that removed that restriction
> (nor can I imagine changes that would do that).  Not sure what the
> preferred approach is regarding coding to handle that invalid data
> transparently or requiring valid data, but we probably should at least
> alert the user that their DB (and, seemingly DB schema) is borked if
> we notice that--whether we fail hard or "just try and see" or insert
> some value in code when reading null from the DB column to try to
> prevent errors.  My personal preference is to not spend a lot of time
> coding to handle invalid data, but to log a message that makes it
> possible for the user to figure out what's wrong and correct it.  That
> said, I'm not at all against bulletproof (or bullet-resistant?) code. 
> (So, I guess that's completely useless advice, and maybe someone else
> can be more helpful. :)

I don't know how the null values occurred. I did not see the user's
database. I am using the services api so maybe it was an empty string.
The xml parser will treat that as null.

I will get a fix for the Guide API to group by callsign and channel
number as you suggested.

Peter