Mailing List Archive

On 8/9/06, toby <toby@np.co.nz> wrote:
> As those who use it know, you get the following entries in the output xml...
> <!--Unknown_Please_Report ID="55" Len="4" -->
> <!--Unknown_Please_Report ID="5f" Len="4" -->
>
> My guess is that either 55 or 5f hold the category data but that this ID
> isn't defined or setup correctly in tv_grab_dvb as category data?

> void parseContentDescription(descr_content_t *dc)
> {
> int c1,c2;
> int i;
> for (i=0;i<(dc->descriptor_length);i+=2)
> {
> nibble_content_t
> *nc=CastContentNibble((char*)dc+DESCR_CONTENT_LEN+i);
>
> c1=((nc->content_nibble_level_1<<4)+(nc->content_nibble_level_2));
>
> c2=((nc->user_nibble_1<<4)+(nc->user_nibble_2));
> if (c1>0)
> printf("\t<category>%s</category>\n",lookup((struct
> lookup_table*)&description_table,c1));
> // This is weird in the uk, they use user but not content,
> and almost the same values
> if (c2>0)
> printf("\t<category>%s</category>\n",lookup((struct
> lookup_table*)&description_table,c2));
> }
> }
>
> Anyone have any ideas on this?

Yes, I've been looking at this. First, note that comment about how in
the UK they use "user_nibble" values instead of "content_nibble"
values. That's applicable here too. I believe the reason Sky are doing
that is because they are not using the standard set of values. For
example that 5f value you've used as an example is not a value defined
by the standard which is available free from etsi.org, though you have
to register. The one you want is EN 300 468. Sky appear to be using
the "content_nibble" values to flag certain types of content. Going by
their website it looks like those values indicate things like
"contains violence" or "pay-per-view". But it doesn't seem to be 100%.

As for the category data, the problem is: how do we figure out what
that 5f represents? Sky don't display categories on the EPG in their
decoders, AFAICS. They do have categories on their website, but only
the top level ones (i.e. the 5 in 5f is defined in the standard as
meaning "children's / youth programs", and that is what the Sky
website shows).

Where we have definitions they seem to be accurate. I.e. the codes
they use appear to mostly correspond to the standard, where they are
defined. For example, the Crocodile Hunter has code 91 which is
"nature/animals/environment" according to the standard.

What I have now is a patch that makes tv_grab_dvb output only the
"user_nibble" codes, and decode them by using the codes from the
standard. Where we don't have a definition it defaults back to the top
level category code (i.e. 5f would be listed as "children's / youth
programs", the same as 50). I have also added a switch to just use the
top level categories as well. However I want to clean up the
tv_grab_dvb code a bit and put the table of mappings into a config
file so people can change the categories if they think they are wrong.

Cheers,
Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

One of them i think was On Wed, 9 Aug 2006 17:54:35 +1200
"toby" <toby@np.co.nz> wrote:

> MessageI've been looking into the missing category situation with DVB EPG stuff.
> It appears the categories are actually in the stream as I think David pointed out in an earlier post.
>
> The issue appears to be the way tv_grab_dvd is working the data.
> I think i've narrowed it down to the parseContentDescription function but i'm not a C expert so can only follow the logic.
>
> As those who use it know, you get the following entries in the output xml...
> <!--Unknown_Please_Report ID="55" Len="4" -->
> <!--Unknown_Please_Report ID="5f" Len="4" -->
>
> My guess is that either 55 or 5f hold the category data but that this ID isn't defined or setup correctly in tv_grab_dvb as category data?
>
>
>
> here is parseContentDescription:
>
> void parseContentDescription(descr_content_t *dc)
> {
> int c1,c2;
> int i;
> for (i=0;i<(dc->descriptor_length);i+=2)
> {
> nibble_content_t *nc=CastContentNibble((char*)dc+DESCR_CONTENT_LEN+i);
> c1=((nc->content_nibble_level_1<<4)+(nc->content_nibble_level_2));
> c2=((nc->user_nibble_1<<4)+(nc->user_nibble_2));
> if (c1>0) printf("\t<category>%s</category>\n",lookup((struct lookup_table*)&description_table,c1));
> // This is weird in the uk, they use user but not content, and almost the same values
> if (c2>0) printf("\t<category>%s</category>\n",lookup((struct lookup_table*)&description_table,c2));
> }
> }
>
> Anyone have any ideas on this?

yes I downloaded the spec a while ago and identified the 55 and 5f categories and printed out the relevant pages of the pdf so I could try and do something about it.

Now to find it again...

IIRC one of them was rating info.



>
> Regards
> Toby Mills
> www.np.co.nz
>
>

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Wed, 9 Aug 2006 19:08:05 +1200
Nick Rout <nick@rout.co.nz> wrote:

>
> One of them i think was On Wed, 9 Aug 2006 17:54:35 +1200
> "toby" <toby@np.co.nz> wrote:
>
> > MessageI've been looking into the missing category situation with DVB EPG stuff.
> > It appears the categories are actually in the stream as I think David pointed out in an earlier post.
> >
> > The issue appears to be the way tv_grab_dvd is working the data.
> > I think i've narrowed it down to the parseContentDescription function but i'm not a C expert so can only follow the logic.
> >
> > As those who use it know, you get the following entries in the output xml...
> > <!--Unknown_Please_Report ID="55" Len="4" -->
> > <!--Unknown_Please_Report ID="5f" Len="4" -->
> >
> > My guess is that either 55 or 5f hold the category data but that this ID isn't defined or setup correctly in tv_grab_dvb as category data?
> >
> >
> >
> > here is parseContentDescription:
> >
> > void parseContentDescription(descr_content_t *dc)
> > {
> > int c1,c2;
> > int i;
> > for (i=0;i<(dc->descriptor_length);i+=2)
> > {
> > nibble_content_t *nc=CastContentNibble((char*)dc+DESCR_CONTENT_LEN+i);
> > c1=((nc->content_nibble_level_1<<4)+(nc->content_nibble_level_2));
> > c2=((nc->user_nibble_1<<4)+(nc->user_nibble_2));
> > if (c1>0) printf("\t<category>%s</category>\n",lookup((struct lookup_table*)&description_table,c1));
> > // This is weird in the uk, they use user but not content, and almost the same values
> > if (c2>0) printf("\t<category>%s</category>\n",lookup((struct lookup_table*)&description_table,c2));
> > }
> > }
> >
> > Anyone have any ideas on this?
>
> yes I downloaded the spec a while ago and identified the 55 and 5f categories and printed out the relevant pages of the pdf so I could try and do something about it.
>
> Now to find it again...
>
> IIRC one of them was rating info.
>
>

Got it! It is an ETSI standard, number 300 468. They are available free after registration at etsi.org, or I will email the pdf to you.

Basically the 55 field is "Parental Rating Descriptor" and 5f is "Private Data Specifier Descriptor"

Details on 55 are:

This descriptor (see table 64) gives a rating based on age and allows for extensions based on other rating criteria.
Table 64: Parental rating descriptor
Syntax No. of bits Identifier
parental_rating_descriptor(){
descriptor_tag 8 uimsbf
descriptor_length 8 uimsbf
for (i=0;i<N;i++){
country_code 24 bslbf
rating 8 uimsbf
}
}

Semantics for the parental rating descriptor:
country_code: This 24-bit field identifies a country using the 3-character code as specified in ISO 3166 [2]. Each
character is coded into 8-bits according to ISO/IEC 8859-1 [5] and inserted in order into the 24-bit field. In the case that
the 3 characters represent a number in the range 900 to 999, then country_code specifies an ETSI defined group of
countries. These allocations are found in ETR 162 [6].
EXAMPLE: United Kingdom has 3-character code "GBR", which is coded as:
'0100 0111 0100 0010 0101 0010'.
rating: This 8-bit field is coded according to table 65, giving the recommended minimum age in years of the end user.
Table 65: Parental rating descriptor, rating
Rating Description
0x00 undefined
0x01 to 0x0F minimum age = rating + 3 years
0x10 to 0xFF defined by the broadcaster
EXAMPLE: 0x04 implies that end users should be at least 7 years old.


Actually thats hard to read when its set out as plain text.

Details on 5f are:

This descriptor is used to identify the specifier of any private descriptors or private fields within descriptors.
Table 67: Private data specifier descriptor
Syntax No. of bits Identifier
private_data_specifier_descriptor(){
descriptor_tag 8 uimsbf
descriptor_length 8 uimsbf
private_data_specifier 32 uimsbf
}
Semantics for the private data specifier descriptor:
private_data_specifier: The assignment of values for this field is given in ETR 162 [6].





_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 8/9/06, Nick Rout <nick@rout.co.nz> wrote:
> > "toby" <toby@np.co.nz> wrote:
> > > As those who use it know, you get the following entries in the output xml...
> > > <!--Unknown_Please_Report ID="55" Len="4" -->
> > > <!--Unknown_Please_Report ID="5f" Len="4" -->
> Basically the 55 field is "Parental Rating Descriptor" and 5f is "Private Data Specifier Descriptor"

I think you'll find that those examples Toby gave were inside
<category></category> tags - they are from the Content Descriptor's
field, not from the descriptor tag. The stuff he's looking for is in
Table 28.

Cheers,
Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Wed, 9 Aug 2006 19:46:52 +1200
"Steve Hodge" <stevehodge@gmail.com> wrote:

> On 8/9/06, Nick Rout <nick@rout.co.nz> wrote:
> > > "toby" <toby@np.co.nz> wrote:
> > > > As those who use it know, you get the following entries in the output xml...
> > > > <!--Unknown_Please_Report ID="55" Len="4" -->
> > > > <!--Unknown_Please_Report ID="5f" Len="4" -->
> > Basically the 55 field is "Parental Rating Descriptor" and 5f is "Private Data Specifier Descriptor"
>
> I think you'll find that those examples Toby gave were inside
> <category></category> tags - they are from the Content Descriptor's
> field, not from the descriptor tag. The stuff he's looking for is in
> Table 28.

well I do not profess to be a c expert, but the message "Unknown_Please_Report ID=..." comes from this bit of code:

tag=*(desc+i) &0xff;
taglen=*(desc+i+1) &0xff;
if (taglen>0)
{
switch (tag)
{
case 0:
break;;
case 0x4D: //short evt desc, [title] [desc]
if (round == 0)
parseEventDescription(desc+i);
break;;
case 0x4E: //long evt descriptor
if (round == 0)
printf("\t<!-- Long Event Info Detected - Bug Author to decode -->\n
");
break;;
case 0x50: //component desc [video] [audio]
if (round == 2)
parseComponentDescription(CastComponentDescriptor(desc+i), 1, &seen)
;
else if (round == 3)
parseComponentDescription(CastComponentDescriptor(desc+i), 0, &seen)
;
break;;
case 0x54: //content desc [category]
if (round == 1)
parseContentDescription(CastContentDescriptor(desc+i));
break;;
case 0x64: //Data broadcast desc - Text Desc for Data components
break;;
default:
if (round == 0)
printf("\t<!--Unknown_Please_Report ID=\"%x\" Len=\"%d\" -->\n",tag,
taglen);

It looks to me that the switch statement is sorting tag values that correspond to table 12 in the spec, and in table 12 the tag values 55 and 5f have the meanings I described earlier. (in the same way that the tag values described in the code (4d, 4e, 50, 54, 64) correspond to entries in Table 12.

But if I am wrong, I am sorry for leading people up the garden path. I don't really understand the code, but would like to!

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 8/9/06, Nick Rout <nick@rout.co.nz> wrote:
> well I do not profess to be a c expert, but the message "Unknown_Please_Report ID=..." comes from this bit of code:

> But if I am wrong, I am sorry for leading people up the garden path. I don't really understand the code, but would like to!

Oops, sorry, you're right. The category code returns "Unknown ID X".
As you can see from that case statement tv_grab_dvb already has some
code to parse content descriptors, but it doesn't decode all the ones
that Sky use.

I did take a look at those two "unknown" descriptors in the Sky data.
The parental rating descriptor does have good data though I haven't
figured out if Myth can use it. The private data specifier descriptior
appears to be set to 0 in all cases I've checked.

Cheers,
Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

This information was provided to me and I was asked to repost the
following to this list, in the hope that it may be of some use to the
community:

------------------------------------------------------------------

The experimentation of another PVR group in NZ a while ago uncovered that
the submitted patches on the darkskiez site solved the issue of unknown
tags.

With a small bit of tweaking the xml became ...

<programme channel="{some channel info}" start="20060821230000"
stop="20060821233000">
<title lang="eng">The Coach Trip</title>
<desc lang="eng">{some description info}</desc>
<category>tourism/travel</category>
<user>0xa1</user>
</programme>

There has also been much use of the attached python script which wraps
the dvbsnoop package.

I place this code into the public domain here, with permission of the
author, if it is of any use. It
seems dvbsnoop is coded to recognise at least the stardard DVB
interpretations and some rating info.

The xml output of this script is something like ...

<programme channel="{some random channel}" start="20060817103000"
stop="20060817230000">
<title lang="eng">The Racing Show</title>
<desc length="197">{description info was here}</desc>
<duration mjd="0xd2cb223000" from="2006-08-17 10:30:00"
to="2006-08-17 23:00:00">00:30:00</duration>
<rating country="NZL" parental="minimum age: 5 years">2</rating>
<content genre="equestrian">0x4a</content>
</programme>

Code as attached.

>This information was provided to me and I was asked to repost the
>following to this list, in the hope that it may be of some use to the
>community:

Andrew, this is a huge help. Thank-you very much, it has solved the mythtv
category problem for good.
I just ran your script on my box and it worked perfectly, not only dumping
out categories but also age restrictions.
Mythtv ate it up perfectly and now I have all the colours of the rainbow in
my guide data.

Yay, thank-you again.

Perhaps the hairy.geek feed could be updated to use this so people without
dvb cards can get it also.

Regards
Toby


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/