Mailing List Archive

On 2/2/06, Corrin Lakeland <lakeland@go.org.nz> wrote:
> Short of buying a DVB card and splitting my digital signal, is there
> any way to get the EPG data out of my sky box?

None I know of. There is a serial port on the back of the decoder but
I've never managed to get anything meaningful out of it.

> What about Sky UHF?

Does the UHF decoder even have an EPG?

> PS: The local free newspaper contains TV listings, so it can't be
> that expensive.

Maybe not if you're publishing it on paper. But, e.g. 3 charge $2000
per month for you to host their data on your website
(http://tv3.co.nz/FAQs/tabid/62/articleID/71/Default.aspx). I doubt
they'd consider this proposal to be any different.

> Would people be interested in contacting sky and
> asking to purchase a legal EPG feed which we could then distribute
> amongst subscribers/ Probably via bittorrent?

I would be, but I suspect the costs would be prohibitive. Sky in
particular now have a huge incentive to keep electronic guide data
locked up: it's one of the major selling points of their PVR.

Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Thu, 2 Feb 2006 18:56:19 +1300
Corrin Lakeland <lakeland@go.org.nz> wrote:

> Hi all,
>
> Short of buying a DVB card and splitting my digital signal, is there
> any way to get the EPG data out of my sky box? I currently have it
> plugged into my PVR500. What about Sky UHF? I know I can download
> the data from the net, I'm looking for something that a C/D letter
> cannot terminate.
>
> PS: The local free newspaper contains TV listings, so it can't be
> that expensive. Would people be interested in contacting sky and
> asking to purchase a legal EPG feed which we could then distribute
> amongst subscribers/ Probably via bittorrent?
>
> Corrin

bittorrent - a waste of time for this amount of data. the mechanics of
downloading the .torrent file every day, then firing up a bittorrent
client to get a 1/2 M data (compressed) is too much bother compared to
good old wget or curl.

I don't think Sky will be giving this away for any affordable price.
Use the existing internet sources is my advice.


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

> bittorrent - a waste of time for this amount of data. the mechanics of
> downloading the .torrent file every day, then firing up a bittorrent
> client to get a 1/2 M data (compressed) is too much bother compared to
> good old wget or curl.

On the other hand, using BitTorrent to get your TV means you get it when the
US gets it, rather than several years later when NZ TV decides to play it.

I notice that Sky 1 have just got up to Season 7 of Stargate SG-1, which I
think TVNZ played for a while, then hid at midnight, then stopped half way
through the season...

Craig


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Thu, 2 Feb 2006 21:39:03 +1300
"Craig Box" <craig@dubculture.co.nz> wrote:

> > bittorrent - a waste of time for this amount of data. the mechanics of
> > downloading the .torrent file every day, then firing up a bittorrent
> > client to get a 1/2 M data (compressed) is too much bother compared to
> > good old wget or curl.
>
> On the other hand, using BitTorrent to get your TV means you get it when the
> US gets it, rather than several years later when NZ TV decides to play it.
>
> I notice that Sky 1 have just got up to Season 7 of Stargate SG-1, which I
> think TVNZ played for a while, then hid at midnight, then stopped half way
> through the season...
>
> Craig


sorry we were discussing the use of bittorrent for transferring EPG
data.

For getting TV shows I thoroughly agree.

Has Battlestar Galactica even played here (past the miniseries shown
around Xmas Day)? I am right up to date on that one :-) (Great show)


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 2/02/2006, at 9:16 PM, Nick Rout wrote:

> On Thu, 2 Feb 2006 18:56:19 +1300
> Corrin Lakeland <lakeland@go.org.nz> wrote:
>
>> Hi all,
>>
>> Short of buying a DVB card and splitting my digital signal, is there
>> any way to get the EPG data out of my sky box? I currently have it
>> plugged into my PVR500.
>>
>> Corrin
>
> bittorrent - a waste of time for this amount of data. the mechanics of
> downloading the .torrent file every day, then firing up a bittorrent
> client to get a 1/2 M data (compressed) is too much bother compared to
> good old wget or curl.


You're probably right. I was thinking wget on the .torrent followed
by btlaunchheadless, but at 500k it would make a very small amount of
data to distribute. My concern is that the generous source providing
this data might not be so willing if hundreds are taking advantage of
it. I know I'm downloading it twice since for some stupid reason
squid is refusing to cache it (I checked the modified header, I'm
pretty sure the fault is at my end, I promise I'll sort it out soon).

> I don't think Sky will be giving this away for any affordable price.
> Use the existing internet sources is my advice.

Okay, how about going back to my first suggestion and buying a cheap
DVB-S card coupled with a PVR150. The DVB card can then extract FTA
channels as well as EPG data, while a PVR150 can pick up the sky
channels (via composite and an IR blaster). A budget DVB-S card is
$160 or less, a PVR150 is $150, while a PVR500 is $270, so you're
barely paying more. Any chance of sky choosing to encrypt the FTA
channels or their EPG data?

PS: Something that might amuse people here, after a lot of
comparisons I finally decided which remote I thought would be best
for controlling my mythtv box. It has "Microsoft" written in large
letters on it... Next think y'know, I'll be installing MCE2k5!

Corrin

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 2006-02-03 at 09:47 +1300, Corrin Lakeland wrote:


> Okay, how about going back to my first suggestion and buying a cheap
> DVB-S card coupled with a PVR150. The DVB card can then extract FTA
> channels as well as EPG data, while a PVR150 can pick up the sky
> channels (via composite and an IR blaster). A budget DVB-S card is
> $160 or less, a PVR150 is $150, while a PVR500 is $270, so you're
> barely paying more. Any chance of sky choosing to encrypt the FTA
> channels or their EPG data?

That would work.

Sky can't encrypt the FTA channels, they aren't Sky ones.

In theory Sky could encrypt the EPG data, but that would mean breaking
the spec, changing the firmware in every decoder or the decoder if need
be.

I don't see Sky doing that anytime soon.

:D


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 2006-02-03 at 09:47 +1300, Corrin Lakeland wrote:


> Okay, how about going back to my first suggestion and buying a cheap
> DVB-S card coupled with a PVR150. The DVB card can then extract FTA
> channels as well as EPG data, while a PVR150 can pick up the sky
> channels (via composite and an IR blaster). A budget DVB-S card is
> $160 or less, a PVR150 is $150, while a PVR500 is $270, so you're
> barely paying more. Any chance of sky choosing to encrypt the FTA
> channels or their EPG data?

That would work.

Sky can't encrypt the FTA channels, they aren't Sky ones.

In theory Sky could encrypt the EPG data, but that would mean breaking
the spec, changing the firmware in every decoder or the decoder if need
be.

I don't see Sky doing that anytime soon.

:D


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

>
>
>>I don't think Sky will be giving this away for any affordable price.
>>Use the existing internet sources is my advice.
>
>Okay, how about going back to my first suggestion and buying a cheap
>DVB-S card coupled with a PVR150. The DVB card can then extract FTA
>channels as well as EPG data, while a PVR150 can pick up the sky
>channels (via composite and an IR blaster). A budget DVB-S card is
>$160 or less, a PVR150 is $150, while a PVR500 is $270, so you're
>barely paying more. Any chance of sky choosing to encrypt the FTA
>channels or their EPG data?
>
>PS: Something that might amuse people here, after a lot of
>comparisons I finally decided which remote I thought would be best
>for controlling my mythtv box. It has "Microsoft" written in large
>letters on it... Next think y'know, I'll be installing MCE2k5!
>
>Corrin
>

Well having a self sufficient EPG source is the main reason I got my
Technisat card. As far as Sky encrypting the EPG, I guess that's a
possibility, but it wouldn't be the easiest thing to implement,
considering the size of the installed user base of old decoders. It
may be something they could do with a firmware update but still a
little bit of a mission. But even if they do encrypt it, it should
be possible to decrypt it (in theory) since the data is small enough
and doesn't have to be done in real time, unlike having to decrypt a
sky digital video stream.

- Wade

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 3 Feb 2006 09:47:24 +1300
Corrin Lakeland wrote:

> You're probably right. I was thinking wget on the .torrent followed
> by btlaunchheadless, but at 500k it would make a very small amount of
> data to distribute. My concern is that the generous source providing
> this data might not be so willing if hundreds are taking advantage of
> it.

Yes a few hundred downloaders could create a problem.

If I was doing it I would upload it to a freely available service, like the homepages that ISP's offer and then let people use the ISP's bandwidth.

> I know I'm downloading it twice since for some stupid reason
> squid is refusing to cache it (I checked the modified header, I'm
> pretty sure the fault is at my end, I promise I'll sort it out soon).
>
> > I don't think Sky will be giving this away for any affordable price.
> > Use the existing internet sources is my advice.
>
> Okay, how about going back to my first suggestion and buying a cheap
> DVB-S card coupled with a PVR150. The DVB card can then extract FTA
> channels as well as EPG data, while a PVR150 can pick up the sky
> channels (via composite and an IR blaster). A budget DVB-S card is
> $160 or less, a PVR150 is $150, while a PVR500 is $270, so you're
> barely paying more. Any chance of sky choosing to encrypt the FTA
> channels or their EPG data?

Thats my plan (I have the PVR-150, soon to get a DVB-S from Europe.)


>
> PS: Something that might amuse people here, after a lot of
> comparisons I finally decided which remote I thought would be best
> for controlling my mythtv box. It has "Microsoft" written in large
> letters on it... Next think y'know, I'll be installing MCE2k5!
>
> Corrin

Rob Fisher uses one of those remottes very successfully with myth, and it is also programmed to control his TV. cool.


--
Nick Rout <nick@rout.co.nz>


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 2/3/06, Wade Maxfield <mythtvnz@hotblack.co.nz> wrote:
> Well having a self sufficient EPG source is the main reason I got my
> Technisat card.

Likewise, though an extra recorder for TV1+2 is handy and the quality
improvement on those two channels is noticable.

> As far as Sky encrypting the EPG, I guess that's a
> possibility, but it wouldn't be the easiest thing to implement,
> considering the size of the installed user base of old decoders. It
> may be something they could do with a firmware update but still a
> little bit of a mission.

I doubt they could do it well without replacing the decoders. The
problem is that the CPU in those decoders is slow - just try using
their EPG to get an idea how bad it is. Decoding 500K of data is going
to be a major burden if the encryption scheme is at all CPU intensive.

> But even if they do encrypt it, it should
> be possible to decrypt it (in theory) since the data is small enough
> and doesn't have to be done in real time, unlike having to decrypt a
> sky digital video stream.

Yeah. I think that Sky are far more likely to go after people
redistributing this data via legal threats than to attempt to encrypt
the EPG data.

Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

> Yeah. I think that Sky are far more likely to go after people
> redistributing this data via legal threats than to attempt to encrypt
> the EPG data.
>

But aren't a lot of the people using the EPG data (apart from the FTA
stuff) already customers of Sky that have paid for the information? I
certainly feel that way and compared to what I used to get from Sky in the
UK (over 100 channels) for an equivalent monthly outlay I'm getting ripped
off anyway (but we won't go there...)

--
Robin Gilks



_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 2006-02-03 at 12:49 +1300, Steve Hodge wrote:
> On 2/3/06, Wade Maxfield <mythtvnz@hotblack.co.nz> wrote:
> > As far as Sky encrypting the EPG, I guess that's a
> > possibility, but it wouldn't be the easiest thing to implement,
> > considering the size of the installed user base of old decoders. It
> > may be something they could do with a firmware update but still a
> > little bit of a mission.
>
> I doubt they could do it well without replacing the decoders. The
> problem is that the CPU in those decoders is slow - just try using
> their EPG to get an idea how bad it is. Decoding 500K of data is going
> to be a major burden if the encryption scheme is at all CPU intensive.

You're assuming it's handled by the CPU. The way the boxes are designed,
all encryption is offloaded to NDS-supplied chipsets, and they already
cope with at least 5Mbit/s of video+audio, so I doubt the EPG data is
going to pose a problem. (EPG data on Sky transponders runs between 0.5
and 1Mbit/s.) The slowness of the rest of the box doesn't reflect any
inablity to encrypt the data, it's only UI rubbish hitting the CPU :)

FWIW, the DVB spec _does_ allow EPG data to be encrypted. Yes, it'll be
a last step, but one they can probably do with little fanfare.

> > But even if they do encrypt it, it should
> > be possible to decrypt it (in theory) since the data is small enough
> > and doesn't have to be done in real time, unlike having to decrypt a
> > sky digital video stream.

It's based on public key crypto. You could try a known plain-text attack
against the data, but that means searching the session key space for
(potentially every packet) each time the key rotates. And I doubt the
public key parts are going to be attackable at all in the near future.

--
David Zanetti <david.zanetti@catalyst.net.nz>
Team Leader, Systems Administration
Catalyst IT Limited
+64-4-8032233 +64-21-402260

On 2/3/06, Robin Gilks <g8ecj@gilks.org> wrote:
> > Yeah. I think that Sky are far more likely to go after people
> > redistributing this data via legal threats than to attempt to encrypt
> > the EPG data.
> >
>
> But aren't a lot of the people using the EPG data (apart from the FTA
> stuff) already customers of Sky that have paid for the information? I
> certainly feel that way and compared to what I used to get from Sky in the
> UK (over 100 channels) for an equivalent monthly outlay I'm getting ripped
> off anyway (but we won't go there...)

Sure, which is another reason I don't think they'll encrypt the data.
But if you start redistributing it to others they'll be less happy.
Even though the only people who can really use the data are paid
subscribers they'll still see it as you getting a benefit from their
data. It doesn't make sense, I know, as the data is essentially
advertising for their pay TV service, but that's how they'll see it.
Plus letting people redistribute the data will weaken their
negotiations with publishers who are currently paying for it.

Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 2/3/06, David Zanetti <david.zanetti@catalyst.net.nz> wrote:
> You're assuming it's handled by the CPU. The way the boxes are designed,
> all encryption is offloaded to NDS-supplied chipsets, and they already
> cope with at least 5Mbit/s of video+audio, so I doubt the EPG data is
> going to pose a problem. (EPG data on Sky transponders runs between 0.5
> and 1Mbit/s.) The slowness of the rest of the box doesn't reflect any
> inablity to encrypt the data, it's only UI rubbish hitting the CPU :)
>
> FWIW, the DVB spec _does_ allow EPG data to be encrypted. Yes, it'll be
> a last step, but one they can probably do with little fanfare.

Video and audio decryption is offloaded, but what are the chances data
decryption can be handled the same way? I guess if the spec allows
encrypted EPG data there is a reasonable chance.

Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On Fri, 2006-02-03 at 14:17 +1300, Steve Hodge wrote:
> On 2/3/06, David Zanetti <david.zanetti@catalyst.net.nz> wrote:
> > FWIW, the DVB spec _does_ allow EPG data to be encrypted. Yes, it'll be
> > a last step, but one they can probably do with little fanfare.
>
> Video and audio decryption is offloaded, but what are the chances data
> decryption can be handled the same way? I guess if the spec allows
> encrypted EPG data there is a reasonable chance.

They're all just PIDs, so there's no technical difference.

Big bit of DVB coming, if you don't care for the details, please skip :)

Everything in the TS you get from the transponder (or a broadcast tower
doing DVB-T, it's all the same) is broken down into seperate streams
identified by PIDs. There's special PIDs like 0x0, containing the
Programme Association Table (tells you where to find the PIDs describing
end-user services), and 0x1 for the Conditional Access Table (what CAs
are in use, and their PIDs for private data), and so forth. Video and
audio are also PIDs, as is the guide data (0x12 for Event Information
Table).

So the standard DVB CA interface is based around telling the decryption
engine to decrypt several PIDs, and arranging the flow of those streams
over the engine. It's possible the engine has a low limit for the number
of PIDs it will decrypt at once, but it should certainly be greater than
2.

In the spec, everything except the system PIDs and tables can be
encrypted. EIT is technically a system PID, but it's explicitly allowed.

IIRC. :)

--
David Zanetti <david.zanetti@catalyst.net.nz>
Team Leader, Systems Administration
Catalyst IT Limited
+64-4-8032233 +64-21-402260

Good expanation of DVB for the uninitiated, thanks David.

Back to EPG alternatives, there is a good screenscraper (ie it downloads web pages and analyses the content into xmltv data) at reven.co.nz.

It is designed for windows, but is written in .NET and runs under linux
with mono. I know it does because I had it working for a while. However
the data throughput is high (a lot of web pages to download) and it also
uses an inordinate amount of memory when running on my mythtv box. I
suspect that it opens all it's current data in RAM and then parses and
sorts it. Whatever the reason, it takes ages and sometimes doesn't
complete. I must try it again, it is often updated. I was being greedy
in requesting 14 days of data at a atime, the author recommends 5. It is
closed source, but the guy is friendly - friendly enough to tweak it to
make it run in mono when a couple of people asked. Unusually for a
(primarily) windows program it runs on the command line.

It also has a handy web based command line builder - choose your
channels, number of days and preferred sources on a web page and it
produces a command line to copy and paste into your terminal or conf
file.


On Fri, 03 Feb 2006 14:45:53 +1300
David Zanetti wrote:

> On Fri, 2006-02-03 at 14:17 +1300, Steve Hodge wrote:
> > On 2/3/06, David Zanetti <david.zanetti@catalyst.net.nz> wrote:
> > > FWIW, the DVB spec _does_ allow EPG data to be encrypted. Yes, it'll be
> > > a last step, but one they can probably do with little fanfare.
> >
> > Video and audio decryption is offloaded, but what are the chances data
> > decryption can be handled the same way? I guess if the spec allows
> > encrypted EPG data there is a reasonable chance.
>
> They're all just PIDs, so there's no technical difference.
>
> Big bit of DVB coming, if you don't care for the details, please skip :)
>
> Everything in the TS you get from the transponder (or a broadcast tower
> doing DVB-T, it's all the same) is broken down into seperate streams
> identified by PIDs. There's special PIDs like 0x0, containing the
> Programme Association Table (tells you where to find the PIDs describing
> end-user services), and 0x1 for the Conditional Access Table (what CAs
> are in use, and their PIDs for private data), and so forth. Video and
> audio are also PIDs, as is the guide data (0x12 for Event Information
> Table).
>
> So the standard DVB CA interface is based around telling the decryption
> engine to decrypt several PIDs, and arranging the flow of those streams
> over the engine. It's possible the engine has a low limit for the number
> of PIDs it will decrypt at once, but it should certainly be greater than
> 2.
>
> In the spec, everything except the system PIDs and tables can be
> encrypted. EIT is technically a system PID, but it's explicitly allowed.
>
> IIRC. :)
>
> --
> David Zanetti <david.zanetti@catalyst.net.nz>
> Team Leader, Systems Administration
> Catalyst IT Limited
> +64-4-8032233 +64-21-402260

--
Nick Rout <nick@rout.co.nz>


_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

On 2/3/06, David Zanetti <david.zanetti@catalyst.net.nz> wrote:
> On Fri, 2006-02-03 at 14:17 +1300, Steve Hodge wrote:
> > Video and audio decryption is offloaded, but what are the chances data
> > decryption can be handled the same way? I guess if the spec allows
> > encrypted EPG data there is a reasonable chance.
>
> They're all just PIDs, so there's no technical difference.

Thanks for the explanation. We'll just have to hope that Sky doesn't
decide to cut us off.

Steve

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/