Mailing List Archive

How to transcode HD recording
When I want to keep a TV recording I usually create a cut list to remove adverts and manually transcode on 'High Quality' (lossless, no resize) to remove them. It works fine, except on a couple of HD recordings I have. The cut list saves and the recordings playback without the adverts but transcode does not actually remove them.
It appears to run, but finishes quite quickly and the cut list is still there.
Some info from backend log is below.....22200_20140102225400.mpg is the file of the recording I'm trying to transcode.
Not sure where it's looking for the tmp file, but my tmp folder and var/tmp folder have enough room for a 13GB file...
I've seen other posts about a similar error, but not any solutions, so thanks for any help!

Jan 3 11:00:04 ....mythbackend: mythbackend[1562]: I Transcode_900 jobqueue.cpp:1991 (DoTranscodeThread) JobQueue: Transcode Starting for .....": High Quality (13.2 GB)
Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
Jan 3 11:00:05 ....mythbackend: mythbackend[1562]: I ProcessRequest mainserver.cpp:1422 (HandleAnnounce) adding: ... as a client (events: 0)
Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest mainserver.cpp:1422 (HandleAnnounce) adding: .... as a client (events: 1)
Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: E ProcessRequest mainserver.cpp:4784 (HandleDeleteFile) Unable to find 22200_20140102225400.mpg.tmp in HandleDeleteFile()
Jan 3 11:00:06 .... mythbackend: mythbackend[1562]: I Transcode_900 jobqueue.cpp:2087 (DoTranscodeThread) JobQueue: Transcode Errored: ....": High Quality (exit status 255, job status was "Errored")
Re: How to transcode HD recording [ In reply to ]
On 03/01/14 13:10, plainface boy wrote:
> When I want to keep a TV recording I usually create a cut list to remove
> adverts and manually transcode on 'High Quality' (lossless, no resize)
> to remove them. It works fine, except on a couple of HD recordings I
> have. The cut list saves and the recordings playback without the adverts
> but transcode does not actually remove them.
> It appears to run, but finishes quite quickly and the cut list is still
> there.
> Some info from backend log is below.....22200_20140102225400.mpg is the
> file of the recording I'm trying to transcode.
> Not sure where it's looking for the tmp file, but my tmp folder and
> var/tmp folder have enough room for a 13GB file...
> I've seen other posts about a similar error, but not any solutions, so
> thanks for any help!
>
> Jan 3 11:00:04 ....mythbackend: mythbackend[1562]: I Transcode_900
> jobqueue.cpp:1991 (DoTranscodeThread) JobQueue: Transcode Starting for
> .....": High Quality (13.2 GB)
> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
> mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
> Jan 3 11:00:05 ....mythbackend: mythbackend[1562]: I ProcessRequest
> mainserver.cpp:1422 (HandleAnnounce) adding: ... as a client (events: 0)
> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
> mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
> mainserver.cpp:1422 (HandleAnnounce) adding: .... as a client (events: 1)
> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: E ProcessRequest
> mainserver.cpp:4784 (HandleDeleteFile) Unable to find
> 22200_20140102225400.mpg.tmp in HandleDeleteFile()
> Jan 3 11:00:06 .... mythbackend: mythbackend[1562]: I Transcode_900
> jobqueue.cpp:2087 (DoTranscodeThread) JobQueue: Transcode Errored:
> ....": High Quality (exit status 255, job status was "Errored")

Other recent threads have been trying to downsize. You are trying to
use lossless cutting, which AIUI works only with mpeg2 recordings. I
may have overlooked clues about what your HD format is, but you may need
another tool.





_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
> On 03/01/14 13:10, plainface boy wrote:
>> When I want to keep a TV recording I usually create a cut list to remove
>> adverts and manually transcode on 'High Quality' (lossless, no resize)
>> to remove them. It works fine, except on a couple of HD recordings I
>> have. The cut list saves and the recordings playback without the adverts
>> but transcode does not actually remove them.
>> It appears to run, but finishes quite quickly and the cut list is still
>> there.
>> Some info from backend log is below.....22200_20140102225400.mpg is the
>> file of the recording I'm trying to transcode.
>> Not sure where it's looking for the tmp file, but my tmp folder and
>> var/tmp folder have enough room for a 13GB file...
>> I've seen other posts about a similar error, but not any solutions, so
>> thanks for any help!
>>
>> Jan 3 11:00:04 ....mythbackend: mythbackend[1562]: I Transcode_900
>> jobqueue.cpp:1991 (DoTranscodeThread) JobQueue: Transcode Starting for
>> .....": High Quality (13.2 GB)
>> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
>> mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
>> Jan 3 11:00:05 ....mythbackend: mythbackend[1562]: I ProcessRequest
>> mainserver.cpp:1422 (HandleAnnounce) adding: ... as a client (events: 0)
>> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
>> mainserver.cpp:1420 (HandleAnnounce) MainServer::ANN Monitor
>> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: I ProcessRequest
>> mainserver.cpp:1422 (HandleAnnounce) adding: .... as a client (events: 1)
>> Jan 3 11:00:05 .... mythbackend: mythbackend[1562]: E ProcessRequest
>> mainserver.cpp:4784 (HandleDeleteFile) Unable to find
>> 22200_20140102225400.mpg.tmp in HandleDeleteFile()
>> Jan 3 11:00:06 .... mythbackend: mythbackend[1562]: I Transcode_900
>> jobqueue.cpp:2087 (DoTranscodeThread) JobQueue: Transcode Errored:
>> ....": High Quality (exit status 255, job status was "Errored")
>
> Other recent threads have been trying to downsize. You are trying to
> use lossless cutting, which AIUI works only with mpeg2 recordings. I
> may have overlooked clues about what your HD format is, but you may need
> another tool.
>
yes - I'm trying to use lossless cutting.
Recording format seems to be;
H264-MPEG-4 AVC, 1920x1080, 25fps, Planar 4:2:0 YUV
so mpeg4.
Is there a way of doing this within mythtv?



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 03.01.2014 21:10, PlainFaceBoy wrote:
>> On 03/01/14 13:10, plainface boy wrote:
>> Other recent threads have been trying to downsize. You are trying to
>> use lossless cutting, which AIUI works only with mpeg2 recordings. I
>> may have overlooked clues about what your HD format is, but you may need
>> another tool.
>>
> yes - I'm trying to use lossless cutting.
> Recording format seems to be;
> H264-MPEG-4 AVC, 1920x1080, 25fps, Planar 4:2:0 YUV
> so mpeg4.
> Is there a way of doing this within mythtv?

mythtranscode only supports lossless cut with MPEG-2 video. But you can
try http://www.mythtv.org/wiki/Lossless_Cut

Regards,
Karl
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 03/01/14 20:26, Karl Dietz wrote:
> On 03.01.2014 21:10, PlainFaceBoy wrote:
>>> On 03/01/14 13:10, plainface boy wrote:
>>> Other recent threads have been trying to downsize. You are trying to
>>> use lossless cutting, which AIUI works only with mpeg2 recordings. I
>>> may have overlooked clues about what your HD format is, but you may need
>>> another tool.
>>>
>> yes - I'm trying to use lossless cutting.
>> Recording format seems to be;
>> H264-MPEG-4 AVC, 1920x1080, 25fps, Planar 4:2:0 YUV
>> so mpeg4.
>> Is there a way of doing this within mythtv?
>
> mythtranscode only supports lossless cut with MPEG-2 video. But you can
> try http://www.mythtv.org/wiki/Lossless_Cut
>
> Regards,
> Karl

Yes: but AIUI it produces an mkv file that it cannot re-edit, as did
this old tool by a former dev:

http://www.mythtv.org/wiki/User:Iamlindoro

There may be others lurking in the wiki or the outside world.

I put a script here that still isn't configured as a myth user job but
works with myth recordings. It's quick and dirty, and is good for
'topping and tailing' but internal cuts may have short audio
discontinuities. It cuts at keyframes, and sometimes, in my HD
recordings, these may be several seconds apart. I find that stepping
forward or back in the editor tends to get almost stuck then, too, but
that may be because of inadequate hardware.

http://www.mythtv.org/wiki/MythDVBcut

John




_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
Quoting John Pilkington <J.Pilk@tesco.net>:
> It cuts at keyframes, and sometimes, in my HD recordings, these may
> be several seconds apart. I find that stepping forward or back in
> the editor tends to get almost stuck then, too, but that may be
> because of inadequate hardware.
>
> http://www.mythtv.org/wiki/MythDVBcut
>

In the UK on the FreeviewHD platform the encoders do statistical
multiplexing ans vary the gaps between keyframes. It's usually very
good at adding keyframes at the start and end of commercial breaks but
can go for over 30 seconds in the end credits of films. Trying to go
back one frame in the cutlist editor can be very slow and appear to
hang. I guess it's also very poor if you are watching live and happen
to tune in those times (even with a normal TV)

Cheers,
Tim.

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 04/01/14 10:14, mythtv@phipps-hutton.freeserve.co.uk wrote:
>
> Quoting John Pilkington <J.Pilk@tesco.net>:
>> It cuts at keyframes, and sometimes, in my HD recordings, these may be
>> several seconds apart. I find that stepping forward or back in the
>> editor tends to get almost stuck then, too, but that may be because of
>> inadequate hardware.
>>
>> http://www.mythtv.org/wiki/MythDVBcut
>>
>
> In the UK on the FreeviewHD platform the encoders do statistical
> multiplexing ans vary the gaps between keyframes. It's usually very good
> at adding keyframes at the start and end of commercial breaks but can go
> for over 30 seconds in the end credits of films. Trying to go back one
> frame in the cutlist editor can be very slow and appear to hang. I guess
> it's also very poor if you are watching live and happen to tune in those
> times (even with a normal TV)
>
> Cheers,
> Tim.

Yes, that fits pretty well with what I see. It's usually possible to
find a good switch point but getting there can be tedious. Sometimes,
too, attempting to step by one keyframe results in no change. I haven't
looked hard at this but it might result from a combination of dead-zones
and close-spaced keyframes.

I've been trying FreeviewHD > DVD to see how much difference I could see
between that and the usual FreeviewSD > DVD. Often not a lot. But
editing is less painful /after/ the necessary (and computationally
expensive) HD > SD conversion, and the result is better.

Incidentally, MythDVBcut on the wiki applies a 188-byte offset that
seemed to give fewer complaints from Project-X but is probably
inappropriate for the non-mpeg2 pycut mode; I'm trying it with this
switch at around line 407.

if ${USEPJX} ; then
cat PXadj$$ | tr "\n" " " > bytelist$$
else
cat PXraw$$ | tr "\n" " " > bytelist$$
fi
echo >> bytelist$$

John



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
I made comments earlier in this thread about wide and variable spacing
of keyframes in FreeviewHD UK programmes. I've been looking at this a
bit more, and my latest test recordings have been better behaved. The
problem certainly exists, but it may be unusual.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 07/01/14 23:29, John Pilkington wrote:
> I made comments earlier in this thread about wide and variable spacing
> of keyframes in FreeviewHD UK programmes. I've been looking at this a
> bit more, and my latest test recordings have been better behaved. The
> problem certainly exists, but it may be unusual.

I have just found that for a recording from DVB-T2 FreeviewHD (h264), a
listing of the markup data generated after mythcommflag --rebuild
contains only a small subset of the lines in a similar listing made
immediately after recording.

In the 'original' table, keyframe separation is typically around 24
frames. In the 'rebuilt' table the spacing is variable, sometimes in
the hundreds. Points listed there are also in the original table.

In my experiments I have often been working with recordings having a
rebuilt seektable. This explains why editing has sometimes been difficult.

Should I open a Ticket?

mythutil --chanid 1102 --starttime 20140109182600 --getmarkup
Haworth_orig.xml
mythcommflag --rebuild --chanid 1102 --starttime 20140109182600
mythutil --chanid 1102 --starttime 20140109182600 --getmarkup
Haworth_rebuilt.xml

ls -l Haw*
-rw-rw-r--. 1 john john 290708 Jan 9 19:13 Haworth_orig.xml
-rw-rw-r--. 1 john john 73286 Jan 9 19:16 Haworth_rebuilt.xml




_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
> On 07/01/14 23:29, John Pilkington wrote:
> > I made comments earlier in this thread about wide and variable spacing
> > of keyframes in FreeviewHD UK programmes. I've been looking at this a
> > bit more, and my latest test recordings have been better behaved. The
> > problem certainly exists, but it may be unusual.
>
> I have just found that for a recording from DVB-T2 FreeviewHD (h264), a
> listing of the markup data generated after mythcommflag --rebuild
> contains only a small subset of the lines in a similar listing made
> immediately after recording.
>
> In the 'original' table, keyframe separation is typically around 24
> frames. In the 'rebuilt' table the spacing is variable, sometimes in
> the hundreds. Points listed there are also in the original table.
>
> In my experiments I have often been working with recordings having a
> rebuilt seektable. This explains why editing has sometimes been difficult.
>
> Should I open a Ticket?
>

How about a 100MB sample cut from beginning of recording using "dd" ?

The UK freeview H264 HD sample clips (I have here) behave well..my
seektables had to be generated by mythcommflag..

Broadcast OTA H264 uses intra-refresh so the peak bitrate is
constrained. All the broadcast samples (I have seen) use this & have
about 2 sec keyframe intervals.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 11/01/14 02:04, HP-mini wrote:
> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
>> On 07/01/14 23:29, John Pilkington wrote:
>>> I made comments earlier in this thread about wide and variable spacing
>>> of keyframes in FreeviewHD UK programmes. I've been looking at this a
>>> bit more, and my latest test recordings have been better behaved. The
>>> problem certainly exists, but it may be unusual.
>>
>> I have just found that for a recording from DVB-T2 FreeviewHD (h264), a
>> listing of the markup data generated after mythcommflag --rebuild
>> contains only a small subset of the lines in a similar listing made
>> immediately after recording.
>>
>> In the 'original' table, keyframe separation is typically around 24
>> frames. In the 'rebuilt' table the spacing is variable, sometimes in
>> the hundreds. Points listed there are also in the original table.
>>
>> In my experiments I have often been working with recordings having a
>> rebuilt seektable. This explains why editing has sometimes been difficult.
>>
>> Should I open a Ticket?
>>
>
> How about a 100MB sample cut from beginning of recording using "dd" ?
>
> The UK freeview H264 HD sample clips (I have here) behave well..my
> seektables had to be generated by mythcommflag..
>
> Broadcast OTA H264 uses intra-refresh so the peak bitrate is
> constrained. All the broadcast samples (I have seen) use this & have
> about 2 sec keyframe intervals.
>

http://code.mythtv.org/trac/ticket/12010

I will attach markup files and a link to a short clip asap.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 11/01/14 12:50, John Pilkington wrote:
> On 11/01/14 02:04, HP-mini wrote:
>> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
>>> On 07/01/14 23:29, John Pilkington wrote:
>>>> I made comments earlier in this thread about wide and variable spacing
>>>> of keyframes in FreeviewHD UK programmes. I've been looking at this a
>>>> bit more, and my latest test recordings have been better behaved. The
>>>> problem certainly exists, but it may be unusual.
>>>
>>> I have just found that for a recording from DVB-T2 FreeviewHD (h264), a
>>> listing of the markup data generated after mythcommflag --rebuild
>>> contains only a small subset of the lines in a similar listing made
>>> immediately after recording.
>>>
>>> In the 'original' table, keyframe separation is typically around 24
>>> frames. In the 'rebuilt' table the spacing is variable, sometimes in
>>> the hundreds. Points listed there are also in the original table.
>>>
>>> In my experiments I have often been working with recordings having a
>>> rebuilt seektable. This explains why editing has sometimes been
>>> difficult.
>>>
>>> Should I open a Ticket?
>>>
>>
>> How about a 100MB sample cut from beginning of recording using "dd" ?
>>
>> The UK freeview H264 HD sample clips (I have here) behave well..my
>> seektables had to be generated by mythcommflag..
>>
>> Broadcast OTA H264 uses intra-refresh so the peak bitrate is
>> constrained. All the broadcast samples (I have seen) use this & have
>> about 2 sec keyframe intervals.
>>
>
> http://code.mythtv.org/trac/ticket/12010
>
> I will attach markup files and a link to a short clip asap.
>
The link to the clip was rejected as 65% probable spam. Let's try here.

http://www.mediafire.com/watch/xeufvnx3etw9umi/1104_20140111121300.mpg


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On Sat, 2014-01-11 at 13:07 +0000, John Pilkington wrote:
> On 11/01/14 12:50, John Pilkington wrote:
> > On 11/01/14 02:04, HP-mini wrote:
> >> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
> >>> On 07/01/14 23:29, John Pilkington wrote:
> >>
> >> How about a 100MB sample cut from beginning of recording using "dd" ?
> >>
> >> The UK freeview H264 HD sample clips (I have here) behave well..my
> >> seektables had to be generated by mythcommflag..
> >>
> >> Broadcast OTA H264 uses intra-refresh so the peak bitrate is
> >> constrained. All the broadcast samples (I have seen) use this & have
> >> about 2 sec keyframe intervals.
> >>
Actually more like every second... it's the hdpvr that is about 2
seconds.

> >
> > http://code.mythtv.org/trac/ticket/12010
> >
> > I will attach markup files and a link to a short clip asap.
> >
>
> http://www.mediafire.com/watch/xeufvnx3etw9umi/1104_20140111121300.mpg
>
At a glance...
mythcommflag in master has the same problem (with your sample)..

Can make it work with "-O FFMPEGTS=1" but then cut editor seeking is
really slow compared to my prodn system.

It works fine (on my prodn system) with
master(v0.28-pre-467-gf600ddf-dirty) last build on Nov 06 2013 with
patches.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 11/01/14 22:24, HP-mini wrote:
> On Sat, 2014-01-11 at 13:07 +0000, John Pilkington wrote:
>> On 11/01/14 12:50, John Pilkington wrote:
>>> On 11/01/14 02:04, HP-mini wrote:
>>>> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
>>>>> On 07/01/14 23:29, John Pilkington wrote:
>>>>
>>>> How about a 100MB sample cut from beginning of recording using "dd" ?
>>>>
>>>> The UK freeview H264 HD sample clips (I have here) behave well..my
>>>> seektables had to be generated by mythcommflag..
>>>>
>>>> Broadcast OTA H264 uses intra-refresh so the peak bitrate is
>>>> constrained. All the broadcast samples (I have seen) use this & have
>>>> about 2 sec keyframe intervals.
>>>>
> Actually more like every second... it's the hdpvr that is about 2
> seconds.
>
>>>
>>> http://code.mythtv.org/trac/ticket/12010
>>>
>>> I will attach markup files and a link to a short clip asap.
>>>
>>
>> http://www.mediafire.com/watch/xeufvnx3etw9umi/1104_20140111121300.mpg
>>
> At a glance...
> mythcommflag in master has the same problem (with your sample)..
>
> Can make it work with "-O FFMPEGTS=1" but then cut editor seeking is
> really slow compared to my prodn system.
>
> It works fine (on my prodn system) with
> master(v0.28-pre-467-gf600ddf-dirty) last build on Nov 06 2013 with
> patches.
>

Just tried mythcommflag -O FFMPEGTS=1 --rebuild and got the short list
again. ffmpeg -version 0.10,9, myth V0.27-131-g4c12503 SL6 i686

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On Sat, 2014-01-11 at 23:30 +0000, John Pilkington wrote:
> On 11/01/14 22:24, HP-mini wrote:
> > On Sat, 2014-01-11 at 13:07 +0000, John Pilkington wrote:
> >> On 11/01/14 12:50, John Pilkington wrote:
> >>> On 11/01/14 02:04, HP-mini wrote:
> >>>> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
> >>>>> On 07/01/14 23:29, John Pilkington wrote:
> >>>>

> > Can make it work with "-O FFMPEGTS=1" but then cut editor seeking is
> > really slow compared to my prodn system.
> >
> > It works fine (on my prodn system) with
> > master(v0.28-pre-467-gf600ddf-dirty) last build on Nov 06 2013 with
> > patches.
> >
>
> Just tried mythcommflag -O FFMPEGTS=1 --rebuild and got the short list
> again. ffmpeg -version 0.10,9, myth V0.27-131-g4c12503 SL6 i686
>

To clarify:
- master (with or without -O FFMPEG=1) fails to find all keyframes
(shortlist).
- there are no IDR keyframes in any broadcast DVB H264 (AFAIK)
- my patched code finds all the keyframes in various international
samples.

Do you build from source ?



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 15/01/14 18:37, HP-mini wrote:
> On Sat, 2014-01-11 at 23:30 +0000, John Pilkington wrote:
>> On 11/01/14 22:24, HP-mini wrote:
>>> On Sat, 2014-01-11 at 13:07 +0000, John Pilkington wrote:
>>>> On 11/01/14 12:50, John Pilkington wrote:
>>>>> On 11/01/14 02:04, HP-mini wrote:
>>>>>> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
>>>>>>> On 07/01/14 23:29, John Pilkington wrote:
>>>>>>
>
>>> Can make it work with "-O FFMPEGTS=1" but then cut editor seeking is
>>> really slow compared to my prodn system.
>>>
>>> It works fine (on my prodn system) with
>>> master(v0.28-pre-467-gf600ddf-dirty) last build on Nov 06 2013 with
>>> patches.
>>>
>>
>> Just tried mythcommflag -O FFMPEGTS=1 --rebuild and got the short list
>> again. ffmpeg -version 0.10,9, myth V0.27-131-g4c12503 SL6 i686
>>
>
> To clarify:
> - master (with or without -O FFMPEG=1) fails to find all keyframes
> (shortlist).
> - there are no IDR keyframes in any broadcast DVB H264 (AFAIK)
> - my patched code finds all the keyframes in various international
> samples.
>
> Do you build from source ?

AFAIK that's the only way of getting 0.27 on el6 clones; so yes, my
system was built from git tarball last weekend. If you're offering, I'd
like to see your patches and ought to be able to apply them. Thanks.
But I still hope Jim will come up with something soon :-)



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On Wed, 2014-01-15 at 18:56 +0000, John Pilkington wrote:
> On 15/01/14 18:37, HP-mini wrote:
> > On Sat, 2014-01-11 at 23:30 +0000, John Pilkington wrote:
> >> On 11/01/14 22:24, HP-mini wrote:
> >>> On Sat, 2014-01-11 at 13:07 +0000, John Pilkington wrote:
> >>>> On 11/01/14 12:50, John Pilkington wrote:
> >>>>> On 11/01/14 02:04, HP-mini wrote:
> >>>>>> On Thu, 2014-01-09 at 20:14 +0000, John Pilkington wrote:
> >>>>>>> On 07/01/14 23:29, John Pilkington wrote:
> >>>>>>
> >
> >
> > To clarify:
> > - master (with or without -O FFMPEG=1) fails to find all keyframes
> > (shortlist).
> > - there are no IDR keyframes in any broadcast DVB H264 (AFAIK)
> > - my patched code finds all the keyframes in various international
> > samples.
> >
> > Do you build from source ?
>
> AFAIK that's the only way of getting 0.27 on el6 clones; so yes, my
> system was built from git tarball last weekend. If you're offering, I'd
> like to see your patches and ought to be able to apply them. Thanks.
> But I still hope Jim will come up with something soon :-)
>

AFAIK The logic around
http://code.mythtv.org/trac/changeset/802e32ba4dc8852da56db170893e95cbc28637e3/mythtv
is odd...
This code change allows any I frame to be treated as keyframe if x
number of frames have processed without finding any..
The code does the opposite of the code patch comment..

AFAIK There are no IDR type keyframes used in broadcast DVB H264.
The IDR causes bitrate spikes.

The H264 parser finds all the real keyframes..for both hd-pvr & normal
broadcast intra-refresh H264..
It always found the IDR frames (if any).

There is a setting for hd-pvr to allow all I frames to be treated as
keyframes. The mpegrecorder sets that variable when using an hd-pvr.

The hd-pvr creates seekable I-frames & IDR keyframes H264 for HD
recordings.

With HD hd-pvr recordings, the decoder can start/seek from all/any I
frames if it has the previous referenced SPS/PPS.

Cutting is a problem using seekable I-frames because you require the
SPS/PPS from a real keyframe to decode.

mkvtoolnix (losscut etc) does not cut on seekable I-frames.

Your recording has a lot of undecode-able data at the start.
Have you tried setting the mythtv-setup option for dvb:
"wait for SEQ start" ?

All H264 & mpeg2video has to start decoding on SEQ or SPS (keyframe).



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: How to transcode HD recording [ In reply to ]
On 15/01/14 20:20, HP-mini wrote:

<snip>
> Your recording has a lot of undecode-able data at the start.
> Have you tried setting the mythtv-setup option for dvb:
> "wait for SEQ start" ?
>
> All H264 & mpeg2video has to start decoding on SEQ or SPS (keyframe).
>

I have 'wait for SEQ header' and 'active EIT scan' enabled. Tuning
delay is 0 ms, but ISTR that that setting doesn't work.

That recording was started unconventionally, because I wanted a short
clip from a sequence transmitted as one piece; so recording of the
programme was enabled, but I had earlier stopped the recording, deleted
it, and erased it from Recorded Programs. Recording then restarted
automatically, and perhaps not as a normal recording would. But I've
just done another 'proper' test and the general look of the first 400
bytes is much the same, with new blocks starting at bytes 0, oxbc, ox178

I'm not sure how long a clip I can send; last time I tried there seemed
to be some sort of threshold at 50 MB.

This is from BBC TWO HD DVB-T2 Wild Brazil, 1102_20140115212800.mpg

dd bs=1M count=48 if=Braz.mpg of=Braz_dd.mpg

http://www.mediafire.com/watch/51w0355pga4bb8w/Braz_dd.mpg
http://www.mediafire.com/view/hl356ki6az461gq/Braz_orig.xml
http://www.mediafire.com/view/mzysobexws83yw6/Braz_rebuilt.xml





_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On Wed, 2014-01-15 at 22:35 +0000, John Pilkington wrote:
> On 15/01/14 20:20, HP-mini wrote:
>
> <snip>

> I'm not sure how long a clip I can send; last time I tried there seemed
> to be some sort of threshold at 50 MB.
>
> This is from BBC TWO HD DVB-T2 Wild Brazil, 1102_20140115212800.mpg
>
> dd bs=1M count=48 if=Braz.mpg of=Braz_dd.mpg
>
> http://www.mediafire.com/watch/51w0355pga4bb8w/Braz_dd.mpg
> http://www.mediafire.com/view/hl356ki6az461gq/Braz_orig.xml
> http://www.mediafire.com/view/mzysobexws83yw6/Braz_rebuilt.xml
>
>

The attached patch appears to generate a usable seektable on your 2
recordings without causing trouble with HDPVR samples or my local
recordings.
The cutlist editor & player seeks cleanly..
I have not confirmed the exact positions with independent tool.
The stream analyzer s/w (I have) is trial/demo & will not run in
virtualbox.
The keyframe positions in the 1st recording clip (simpsons) do seem odd.

The patch just flips around the keyframe search order in
avformatdecoder.. The other changes are mostly just cleanup.

YMMV..
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On 23/01/14 06:19, HP-mini wrote:
> On Wed, 2014-01-15 at 22:35 +0000, John Pilkington wrote:
>> On 15/01/14 20:20, HP-mini wrote:
>>
>> <snip>
>
>> I'm not sure how long a clip I can send; last time I tried there seemed
>> to be some sort of threshold at 50 MB.
>>
>> This is from BBC TWO HD DVB-T2 Wild Brazil, 1102_20140115212800.mpg
>>
>> dd bs=1M count=48 if=Braz.mpg of=Braz_dd.mpg
>>
>> http://www.mediafire.com/watch/51w0355pga4bb8w/Braz_dd.mpg
>> http://www.mediafire.com/view/hl356ki6az461gq/Braz_orig.xml
>> http://www.mediafire.com/view/mzysobexws83yw6/Braz_rebuilt.xml
>>
>>
>
> The attached patch appears to generate a usable seektable on your 2
> recordings without causing trouble with HDPVR samples or my local
> recordings.
> The cutlist editor & player seeks cleanly..
> I have not confirmed the exact positions with independent tool.
> The stream analyzer s/w (I have) is trial/demo & will not run in
> virtualbox.
> The keyframe positions in the 1st recording clip (simpsons) do seem odd.
>
> The patch just flips around the keyframe search order in
> avformatdecoder.. The other changes are mostly just cleanup.
>
> YMMV..
>

Noted, thanks, I'll try a new build.

It's perhaps worth noting that the short seektable appears to do
adequately its main job of controlling skips. It just doesn't find all
potential cutpoints....

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On 23/01/14 09:44, John Pilkington wrote:

>
> Noted, thanks, I'll try a new build.
>

Patched build of 0.27-fixes is working fine for me. I've updated the
ticket. Thank you!

John


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On 23/01/14 23:53, John Pilkington wrote:
> On 23/01/14 09:44, John Pilkington wrote:
>
>>
>> Noted, thanks, I'll try a new build.
>>
>
> Patched build of 0.27-fixes is working fine for me. I've updated the
> ticket. Thank you!
>


The patched code works but the revised explanation seems to me to suffer
from too many and conflicting negatives. I think this is clearer - and
probably explains what is happening: :-)

+ // H.264 recordings from an HD-PVR contain IDR-keyframes,
+ // When these are present they are the only valid cut points +
// for lossless cuts.
+ // DVB-S(2)/T(2) H.264 recordings typically do not contain
+ // IDR-keyframes & use non-IDR I-frames as keyframes.
+ // Initially we look for non-IDR I-frames.
+ // If we get far enough into the rebuild without having
+ // created any seektable entries, we assume that this
+ // setting is wrong, and so we rewind and
+ // look for IDR-keyframes as used by the HD-PVR.

Cheers,

John




_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On Fri, 2014-01-24 at 21:39 +0000, John Pilkington wrote:
> On 23/01/14 23:53, John Pilkington wrote:
> > On 23/01/14 09:44, John Pilkington wrote:
> >
> >>
> >> Noted, thanks, I'll try a new build.
> >>
> >
> > Patched build of 0.27-fixes is working fine for me. I've updated the
> > ticket. Thank you!
> >
>
>
> The patched code works but the revised explanation seems to me to suffer
> from too many and conflicting negatives. I think this is clearer - and
> probably explains what is happening: :-)
>
> + // H.264 recordings from an HD-PVR contain IDR-keyframes,
> + // When these are present they are the only valid cut points +
> // for lossless cuts.
> + // DVB-S(2)/T(2) H.264 recordings typically do not contain
> + // IDR-keyframes & use non-IDR I-frames as keyframes.
> + // Initially we look for non-IDR I-frames.
> + // If we get far enough into the rebuild without having
> + // created any seektable entries, we assume that this
> + // setting is wrong, and so we rewind and
> + // look for IDR-keyframes as used by the HD-PVR.
>
> Cheers,
>
> John
>
AFAIK Broadcast DVB H264 does not use IDR or I frames as keyframes.
Keyframes are invariably periodic intra-refresh B frame sequences
signalled by SEI messages. There can be I slices but they are not
necessarily keyframes..

The "rewind & try again" logic should never be needed for HD-PVR or DVB
H264. It makes some sense to allow I frames as keyframes iff nothing
else can be detected/found.

The "first pass" thru' parser finds any IDR & SEI signalled keyframes...


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
Quoting John Pilkington <J.Pilk@tesco.net>:
> On 23/01/14 06:19, HP-mini wrote:
>> The patch just flips around the keyframe search order in
>> avformatdecoder.. The other changes are mostly just cleanup.
>
> Noted, thanks, I'll try a new build.
>
> It's perhaps worth noting that the short seektable appears to do
> adequately its main job of controlling skips. It just doesn't find
> all potential cutpoints....
>

Just a me too. Prior to the patch I had 3400 keyframes in the
filemarkup table for a 2 hour film. After the patch it went to 8800.
I'm happy for the patch to go in but I only ever run mythcommflag on
H.264 material that is recorded from DVB-T2 and copied to mythvideo.
There are obviously lots of ways of getting H.264 material into
mythvideo so the patch may not be suitable in the general case.
Perhaps an option to mythcommflag would work?

Cheers,
Tim.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users
Re: mythcommflag prob (was How to transcode HD recording) [ In reply to ]
On 29/01/14 15:08, mythtv@phipps-hutton.freeserve.co.uk wrote:
> Quoting John Pilkington <J.Pilk@tesco.net>:
>> On 23/01/14 06:19, HP-mini wrote:
>>> The patch just flips around the keyframe search order in
>>> avformatdecoder.. The other changes are mostly just cleanup.
>>
>> Noted, thanks, I'll try a new build.
>>
>> It's perhaps worth noting that the short seektable appears to do
>> adequately its main job of controlling skips. It just doesn't find
>> all potential cutpoints....
>>
>
> Just a me too. Prior to the patch I had 3400 keyframes in the filemarkup
> table for a 2 hour film. After the patch it went to 8800. I'm happy for
> the patch to go in but I only ever run mythcommflag on H.264 material
> that is recorded from DVB-T2 and copied to mythvideo. There are
> obviously lots of ways of getting H.264 material into mythvideo so the
> patch may not be suitable in the general case. Perhaps an option to
> mythcommflag would work?
>
> Cheers,
> Tim.

In principle perhaps the identification of nearby keyframes could be
done by the editor; but I don't expect to be trying to code that :-)


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://www.mythtv.org/mailman/listinfo/mythtv-users