Mailing List Archive

On 16/12/12 22:23, D. R. Newman wrote:
> On 26/10/12 09:22, Tim Phipps wrote:
>
>> My contribution to MythTV is here:
>>
>> http://www.mythtv.org/wiki/Commercial_detection_with_silence_for_UK_freeviewHD

> I tried this, it often works well, but it is not doing the same as
> mythcommflag_wrapper, as it ignores the minimum time for silence.

So I have written mausc2.c (attached) which takes the minimum silence
into the logic.

However, I wonder if it would be better to simply spit out of mausc the
periods of silence, then do the logic working out which bits are adverts
and which bits are programme in python?

David Newman

On 16/12/12 23:07, D. R. Newman wrote:
>
> So I have written mausc2.c (attached) which takes the minimum silence
> into the logic.

Which has a bug, so here is a corrected version.

Quoting "D. R. Newman" <d.r.newman@e-consultation.org>:

> On 16/12/12 23:07, D. R. Newman wrote:
>>
>> So I have written mausc2.c (attached) which takes the minimum silence
>> into the logic.
>
> Which has a bug, so here is a corrected version.

Thanks for that, I finally got around to testing it and have now
updated the wiki page.

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On Mon, 11 Feb 2013 20:31:18 -0000, <phipps-hutton@sky.com> wrote:

>
> Quoting "D. R. Newman" <d.r.newman@e-consultation.org>:
>
>> On 16/12/12 23:07, D. R. Newman wrote:
>>>
>>> So I have written mausc2.c (attached) which takes the minimum silence
>>> into the logic.
>>
>> Which has a bug, so here is a corrected version.
>
> Thanks for that, I finally got around to testing it and have now updated
> the wiki page.
>
> Cheers,
> Tim.
>
>

Whilst you're updating it, I had a couple of comments that I forgot to
mention earlier;

1. Why do you set avconv to debug output ("-v 8") ? It seemed to run twice
as fast with no adverse effects when I turned it off.

2. The 'comflagged' states are: 0=not flagged, 1=flagged, 2=in progress,
3=commfree.

Thanks for the neat idea though. I wanted to try out some clustering
algorithms to get rid of the rogue silences & your approach provided the
ideal base. I'm still tuning/testing but will add it to the wiki soon.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

Quoting Roger Siddons <dizygotheca@ntlworld.com>:
> Whilst you're updating it, I had a couple of comments that I forgot
> to mention earlier;
>
> 1. Why do you set avconv to debug output ("-v 8") ? It seemed to run
> twice as fast with no adverse effects when I turned it off.
>
Accidently left in from debugging, now changed to "-v fatal".

> 2. The 'comflagged' states are: 0=not flagged, 1=flagged, 2=in
> progress, 3=commfree.

Do you (or anyone listening) know how to get the frontends to update
the skiplist if they are watching the same program? I have to stop
playback and start again to reload the skiplist.

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On 12/02/2013 10:58, phipps-hutton@sky.com wrote:
>
> Quoting Roger Siddons <dizygotheca@ntlworld.com>:
>> Whilst you're updating it, I had a couple of comments that I forgot to
>> mention earlier;
>>
>> 1. Why do you set avconv to debug output ("-v 8") ? It seemed to run
>> twice as fast with no adverse effects when I turned it off.
>>
> Accidently left in from debugging, now changed to "-v fatal".
>
>> 2. The 'comflagged' states are: 0=not flagged, 1=flagged, 2=in
>> progress, 3=commfree.
>
> Do you (or anyone listening) know how to get the frontends to update the
> skiplist if they are watching the same program? I have to stop playback
> and start again to reload the skiplist.

I'd love to see this working as well. My C isn't great and I've never
looked far into the mythtv codebase but it would appear that
mythcommflag sends a "message" that somehow gets to the player:

In programs/mythcommflag/main.cpp:

static void commDetectorGotNewCommercialBreakList(void)
{
frm_dir_map_t newCommercialMap;
commDetector->GetCommercialBreakList(newCommercialMap);

frm_dir_map_t::Iterator it = newCommercialMap.begin();
QString message = "COMMFLAG_UPDATE ";
message += global_program_info->MakeUniqueKey();

for (it = newCommercialMap.begin();
it != newCommercialMap.end(); ++it)
{
if (it != newCommercialMap.begin())
message += ",";
else
message += " ";
message += QString("%1:%2").arg(it.key())
.arg(*it);
}

LOG(VB_COMMFLAG, LOG_INFO,
QString("mythcommflag sending update: %1").arg(message));

gCoreContext->SendMessage(message);
}

and in libs/libmythtv/tv_play.cpp (the only other place in the
mythcodebase that "COMMFLAG_UPDATE" is mentioned based on my grep of the
.cpp files in the code:

if (message.left(15) == "COMMFLAG_UPDATE" && (tokens.size() >= 3))
{
uint evchanid = 0;
QDateTime evrecstartts;
ProgramInfo::ExtractKey(tokens[1], evchanid, evrecstartts);

PlayerContext *mctx = GetPlayerReadLock(0, __FILE__, __LINE__);
for (uint i = 0; mctx && evchanid && (i < player.size()); i++)
{
PlayerContext *ctx = GetPlayer(mctx, i);
ctx->LockPlayingInfo(__FILE__, __LINE__);
bool doit =
((ctx->playingInfo) &&
(ctx->playingInfo->GetChanID() == evchanid) &&
(ctx->playingInfo->GetRecordingStartTime() ==
evrecstartts));
ctx->UnlockPlayingInfo(__FILE__, __LINE__);


I'm not sure if this helps, but if there was some way to "send a
message" into the mythtv infrastructure and you could work out the
message format then maybe it could be integrated.

Rob.

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On 11/02/13 20:31, phipps-hutton@sky.com wrote:
>
> Quoting "D. R. Newman" <d.r.newman@e-consultation.org>:
>
>> On 16/12/12 23:07, D. R. Newman wrote:
>>>
>>> So I have written mausc2.c (attached) which takes the minimum silence
>>> into the logic.
>
> Thanks for that, I finally got around to testing it and have now updated
> the wiki page.

BTW, I did try to write a program based on mausc.c that produces
statistics of silences in a program, so that if lots of people ran it on
recordings from different channels, we could find out the optimum gap
and depth of silence to count as a marker between adverts, and a longer
gap as a marker of the beginning and end of the a group of adverts.

But I couldn't get it to work properly, so I left it when I had
somethign else to do. If anyone is interested in getting it to work,
please let me know.

David Newman
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On Tue, 12 Feb 2013 10:58:03 -0000, <phipps-hutton@sky.com> wrote:

> Do you (or anyone listening) know how to get the frontends to update the
> skiplist if they are watching the same program? I have to stop playback
> and start again to reload the skiplist.
>

I finally found time to get this working using the commflag code that
Robert highlighted.
Here's some code fragments that should enable you to do the same...

#include <mythcorecontext.h>
#include <mythcontext.h>
#include <mythversion.h>
#include <programtypes.h>
#include <QCoreApplication>

// Player update message
QString updateMessage;
char message[200];

void UpdatePlayer(frame_t start, frame_t stop) {

// add comma unless it's first cut
if (!updateMessage.endsWith(' '))
updateMessage += ',';

updateMessage += QString("%1:%2,%3:%4")
.arg(start).arg(MARK_COMM_START)
.arg(stop).arg(MARK_COMM_END);

// printf("debug@Sending %s\n", updateMessage.toLatin1().data());

gCoreContext->SendMessage(updateMessage);
}

int main(int argc, char **argv) {

QCoreApplication a(argc, argv);
QCoreApplication::setApplicationName("mausc");

MythContext* gContext = new MythContext(MYTH_BINARY_VERSION);

if (!gContext->Init( false, /*use gui*/
false, /*prompt for backend*/
false, /*bypass auto discovery*/
false)) /*ignoreDB*/
{
// printf("err%cContext initialisation failed\n", DELIMITER);
exit(1);
}

gCoreContext->ConnectToMasterServer();

-----------

if (1 != sscanf(argv[5], "%s", message)) {
// report("err", ' ', "Could not parse message option into a
string", 0, 0, 0, 0);
usage(argv[0]);
exit(2);
}
updateMessage = "COMMFLAG_UPDATE " + QString(message) + ' ';

---------

Requires a new arg from mausc-wrapper.py:

# player update message needs prog id
progId = str(chanid) + '_' + str(starttime)

----------

My Makefile:

INCPATH = -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtNetwork
-I/usr/include/qt4/QtSql -I/usr/include/qt4 -I/usr/include/mythtv
LIBPATH = -L/usr/lib -L/usr/lib/i386-linux-gnu

all: mausc catagrower

mausc: mausc.c
g++ $(INCPATH) $(LIBPATH) mausc.c -o /usr/local/bin/mausc -lsndfile
-lQtCore -lmythbase-0.26 -lmyth-0.26

catagrower: catagrower.c
g++ catagrower.c -o /usr/local/bin/catagrower

--------

Good Luck!
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On Fri, 26 Oct 2012 09:22:08 +0100, Tim Phipps <phipps-hutton@sky.com>
wrote:

> My contribution to MythTV is here:
>
> http://www.mythtv.org/wiki/Commercial_detection_with_silence_for_UK_freeviewHD
>
> It's based on the one above but works on DVB-T2 as well, does not need
> to transcode the audio stream and can be run in realtime. I've not
> figured out how to get it to update the frontends as it progresses yet
> (help needed).
>
> It works on all the Freeview channels I receive but don't try it on
> cartoons/animation they have silences that confuse it.
>

Hope you don't mind that I've added my 'improved' version to your Wiki
page.
It's tunable and does a reasonable job of animations for me.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

Quoting Roger Siddons <dizygotheca@ntlworld.com>:


> Hope you don't mind that I've added my 'improved' version to your Wiki page.
> It's tunable and does a reasonable job of animations for me.

Great stuff Roger though I think it's your Wiki page now ;-)

I get a segfualt at the end that is fixed with this patch:

--- /bfd/snapshots/hourly.0/mythtv/bin/silence.cpp 2013-03-04
19:26:57.000000000 +0000
+++ silence.cpp 2013-03-05 10:05:32.490654368 +0000
@@ -414,9 +414,10 @@
currentCluster.restart(thisSilence);
}
// log silence
- report("debug", thisSilence->state_log[thisSilence->state], "Silence",
- thisSilence->start, thisSilence->end,
- thisSilence->interval, thisSilence->power);
+ if (thisSilence)
+ report("debug", thisSilence->state_log[thisSilence->state],
"Silence",
+ thisSilence->start, thisSilence->end,
+ thisSilence->interval, thisSilence->power);
}

I've not used this very long but is it waiting for the next set of
silences before recording the last advert break? It just seems to take
longer to record a break than I was expecting.

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On Tue, 05 Mar 2013 10:20:38 -0000, <phipps-hutton@sky.com> wrote:

> I get a segfualt at the end that is fixed with this patch:
>

Good spot. What logging are you using? Have you tweaked it further? And
how did you detect this ?
It occurs on the last line and doesn't affect functionality but I'm
concerned that I don't see any indication of this on my system, even
though it must be segfaulting every time.

>
> I've not used this very long but is it waiting for the next set of
> silences before recording the last advert break? It just seems to take
> longer to record a break than I was expecting.
>

It does. On a typical programme, this design will only flag the 1st advert
when it reaches the 2nd. It seemed elegant when only comflagging completed
recordings, which is what I did until a couple of days ago.
I rarely watch in-progress recordings and only tacked this on as a result
of our recent discussions. So it is currently rubbish at flagging
in-progress recordings!
I'm currently re-designing it to work effectively - v2 ETA ~1 week.

Moreover, I noticed yesterday that a comflagging job still doesn't flag a
programme that you're watching (for my shoddy testing I only ran the
script manually). I was hoping to get v2 out before anyone noticed, but
the fix for that is to change silence.py #156 to:

# player update message needs prog id (with time in Qt::ISODate format)
progId = str(chanid) + '_' + str(starttime).replace(' ','T')
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users

On 6 March 2013 02:50, Roger Siddons <dizygotheca@ntlworld.com> wrote:

> On Tue, 05 Mar 2013 10:20:38 -0000, <phipps-hutton@sky.com> wrote:
>
> I get a segfualt at the end that is fixed with this patch:
>>
>>
> Good spot. What logging are you using? Have you tweaked it further? And
> how did you detect this ?
> It occurs on the last line and doesn't affect functionality but I'm
> concerned that I don't see any indication of this on my system, even though
> it must be segfaulting every time.
>
>
>
>> I've not used this very long but is it waiting for the next set of
>> silences before recording the last advert break? It just seems to take
>> longer to record a break than I was expecting.
>>
>>
> It does. On a typical programme, this design will only flag the 1st advert
> when it reaches the 2nd. It seemed elegant when only comflagging completed
> recordings, which is what I did until a couple of days ago.
> I rarely watch in-progress recordings and only tacked this on as a result
> of our recent discussions. So it is currently rubbish at flagging
> in-progress recordings!
> I'm currently re-designing it to work effectively - v2 ETA ~1 week.
>
> Moreover, I noticed yesterday that a comflagging job still doesn't flag a
> programme that you're watching (for my shoddy testing I only ran the script
> manually). I was hoping to get v2 out before anyone noticed, but the fix
> for that is to change silence.py #156 to:
>
> # player update message needs prog id (with time in Qt::ISODate format)
> progId = str(chanid) + '_' + str(starttime).replace(' ','T')
>
>
> Hi Roger,

how did you determin your silence.presets?

trial and error or do you have some script you can run over recordings?

I find this works fine with the defaults on most Australian channels but
there are a few which it detects no ad's. ie.Nine Digital

Cheers,

Anthony

On 6 March 2013 21:45, Anthony Giggins <seven@seven.dorksville.net> wrote:

> On 6 March 2013 02:50, Roger Siddons <dizygotheca@ntlworld.com> wrote:
>
>> On Tue, 05 Mar 2013 10:20:38 -0000, <phipps-hutton@sky.com> wrote:
>>
>> I get a segfualt at the end that is fixed with this patch:
>>>
>>>
>> Good spot. What logging are you using? Have you tweaked it further? And
>> how did you detect this ?
>> It occurs on the last line and doesn't affect functionality but I'm
>> concerned that I don't see any indication of this on my system, even though
>> it must be segfaulting every time.
>>
>>
>>
>>> I've not used this very long but is it waiting for the next set of
>>> silences before recording the last advert break? It just seems to take
>>> longer to record a break than I was expecting.
>>>
>>>
>> It does. On a typical programme, this design will only flag the 1st
>> advert when it reaches the 2nd. It seemed elegant when only comflagging
>> completed recordings, which is what I did until a couple of days ago.
>> I rarely watch in-progress recordings and only tacked this on as a result
>> of our recent discussions. So it is currently rubbish at flagging
>> in-progress recordings!
>> I'm currently re-designing it to work effectively - v2 ETA ~1 week.
>>
>> Moreover, I noticed yesterday that a comflagging job still doesn't flag a
>> programme that you're watching (for my shoddy testing I only ran the script
>> manually). I was hoping to get v2 out before anyone noticed, but the fix
>> for that is to change silence.py #156 to:
>>
>> # player update message needs prog id (with time in Qt::ISODate
>> format)
>> progId = str(chanid) + '_' + str(starttime).replace(' ','T')
>>
>>
>> Hi Roger,
>
> how did you determin your silence.presets?
>
> trial and error or do you have some script you can run over recordings?
>
> I find this works fine with the defaults on most Australian channels but
> there are a few which it detects no ad's. ie.Nine Digital
>
> Cheers,
>
> Anthony
>
> ignore me just saw the line just below it in the wiki

"Once you understand the logging information you can easily tune your own
channels/programmes by experimenting with the --preset option directly from
a command line until you get decent results. For example;

silence.py --chanid 1004 --starttime 20130117220000 --loglevel debug
--preset "-80,,3,,180"
"

Cheers,

Anthony

Quoting Roger Siddons <dizygotheca@ntlworld.com>:

> On Tue, 05 Mar 2013 10:20:38 -0000, <phipps-hutton@sky.com> wrote:
>
>> I get a segfualt at the end that is fixed with this patch:
>>
>
> Good spot. What logging are you using? Have you tweaked it further?
> And how did you detect this ?
> It occurs on the last line and doesn't affect functionality but I'm
> concerned that I don't see any indication of this on my system, even
> though it must be segfaulting every time.

I am weird enough to have 'dmesg -c' running in an hourly cron job so
I get a nice email whenever something bad happens (unless it's bad
enough to take out the email system).

>>
>> I've not used this very long but is it waiting for the next set of
>> silences before recording the last advert break? It just seems to
>> take longer to record a break than I was expecting.
>>
>
> It does. On a typical programme, this design will only flag the 1st
> advert when it reaches the 2nd. It seemed elegant when only
> comflagging completed recordings, which is what I did until a couple
> of days ago.
> I rarely watch in-progress recordings and only tacked this on as a
> result of our recent discussions. So it is currently rubbish at
> flagging in-progress recordings!
> I'm currently re-designing it to work effectively - v2 ETA ~1 week.

OK, will wait with bated breath.

>
> Moreover, I noticed yesterday that a comflagging job still doesn't
> flag a programme that you're watching (for my shoddy testing I only
> ran the script manually). I was hoping to get v2 out before anyone
> noticed, but the fix for that is to change silence.py #156 to:
>
> # player update message needs prog id (with time in Qt::ISODate format)
> progId = str(chanid) + '_' + str(starttime).replace(' ','T')

Thanks.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users