Mailing List Archive

I have two sources; OTA via an antenna and dual tuner and two cable STBs
via firewire on a master backend and a slave backend. By looking at the
recordings with ffprobe, I have determined that the latter (cable) is
h264 while the former all seems to be mpeg2video and a much higher
quality and larger file sizes.

I have had this setup for years and, until V.31, all worked well.  Now,
the cable recordings (which are becoming fewer all the time as I improve
my antenna system) have trouble.  Occasionally, some attempts at
playback cause the frontend to fail with "Decoder Error" and, less
often, even crash the frontend.  Also, mythcommflag hangs at times on
the h264 recordings and rarely finds commercials accurately.  The OTA
recordings (mpeg2video) never have trouble with playback or commercial
flagging.  These problems are discussed in this bug report/discussion:
https://code.mythtv.org/trac/ticket/13557#comment:13

Unfortunately, I also switched from Nvidia/VDPAU to Intel/VAAPI at the
same time I upgraded to V.31, so I'm not sure how much of my problems
are specific to VAAPI, but the commflag problems are unrelated to
playback, so I suspect it is the highly compressed h264 recordings not
fitting into the new playback mechanism.

Anyway, the above-mentioned thread gives instructions on how to use dd
to removed the beginning of the recording, which renders a recording
that formerly crashed the frontend playable.  I have repaired three
recordings, so far, and it has worked perfectly each time.  I have
incorporated the technique into a script and included it, below.  I
found that ffprobe was giving me durations that were MUCH higher that
they should be (one 2-hour show reported an 11 hour duration) so, since
I didn't know what else to look for in ffprobe's output, I used that
duration and compared it with what should have been, based on a database
query.  If the duration is more than .2 hours longer than the database
time, I run dd, strip off the beginning of the file, then rebuild the
seektable with mythcommflag (per the above-mentioned bug report).

Speaking of ffprobe's output:  the h264 recordings all start with a
series of error messages:

[h264 @ 0x557c4ec5aa40] SPS unavailable in decode_picture_timing
[h264 @ 0x557c4ec5aa40] non-existing PPS 0 referenced
[h264 @ 0x557c4ec5aa40] SPS unavailable in decode_picture_timing
[h264 @ 0x557c4ec5aa40] non-existing PPS 0 referenced
[h264 @ 0x557c4ec5aa40] decode_slice_header error
[h264 @ 0x557c4ec5aa40] no frame!

This repeats over and over (sent to stderr) anywhere from 20 to 80 (or
more?) times before it finally gives normal (although sometimes
inaccurate) clip information.  I'm not sure what to make of it and
didn't really feel like studying ffprobe to figure it out.

Here's that script.  I run it with cron every AM at 3:30.  I assume
there will be no more "Decoder Error" messages/crashes.

Dave D.


#!/usr/bin/perl -w

use strict;
use DBI;

my @flist;
my $days = 3;
my $files_checked = 0;
my $bad_files = 0;

# run this script with a filename argument(s) to fix a known problem file(s)
# run without arguments to check all recordings newer than 3 days

if ($#ARGV >= 0) {
    @flist = @ARGV;
}
else {
    @flist = `find /shome/mythtv /e/shome/mythtv -iname "*.ts"
-daystart -mmin +120 -mtime -$days`;
    chomp @flist;
}

exit if $#flist < 0;

my $dsns = "dbi:mysql:host=localhost;database=mythconverg";
my %attribs = ('RaiseError', '1'); #, 'AutoCommit', '0');
my $dbh;

$dbh = DBI->connect($dsns, 'root', '', \%attribs);

die "database connect error\n" if (!$dbh);

foreach my $f (@flist) {

    my $chan_ID = 0;
    my $fname = '';

    # /e/shome/mythtv/4115_20200705041000.ts
    #print "$f\n";

    # extract the chanid and recording file basename from full path
    if ($f =~ /\/(\d{4})(_\d{14}.ts)$/) {
        $chan_ID = $1;
        $fname = $1 . $2;
        #print "Channel ID is $chan_ID on $f\n";
    }
    else {
        die "Could not get chanid on $f\n";
    }

    # only process recordings from cable box channels, sourceid = 4
    # this query could be eliminated if all recordings are suspect
    my $query = "SELECT sourceid FROM channel WHERE chanid = $chan_ID";
    my $sth = $dbh->prepare($query);
    $sth->execute();
    my $rowref = $sth->fetchrow_arrayref;
    $sth->finish;
    next if $rowref->[0] != 4;

    # select time_to_sec(timediff('2010-09-01 03:00:00', '2010-09-01
00:10:00' )) / 3600;
    # $start_time was going to indicate if recording was finished, but
I handled that with -mmin +120 in the find command, above
    $query = "SELECT starttime, time_to_sec(timediff(endtime,
starttime)) / 3600 FROM recorded WHERE basename = '$fname'";
    $sth = $dbh->prepare($query);
    $sth->execute();
    $rowref = $sth->fetchrow_arrayref;
    my $start_time = $rowref->[0];
    my $db_hours = $rowref->[1];
    $sth->finish;

    $files_checked++;

    my $info = `/usr/bin/ffprobe $f 2>&1 | grep Duration:`;
    chomp $info;

    # "  Duration: 00:29:55.54, start: 11163.435322, bitrate: 3892 kb/s"
    #print "File info for $f is \"$info\"\n";
    if ($info =~ /Duration: (\d\d):(\d\d):(\d\d)\.\d\d/) {
        #print "Duration was $1 hours, $2 minutes\n";
        my $hrs = $1;
        my $mins = $2;
        my $secs = $3;
        my $hours = $hrs + ($mins / 60.0) + ($secs / 3.6e2); # convert
$hours to floating point and add fraction from $mins

        #printf "%1.3f\n", $hours - $db_hours; # typical file size
bigger than DB size
        if ($hours > $db_hours + 0.2) { # typical file size bigger than
DB size by about 0.11 hours
            printf "Database: %1.3f   ffprobe: %1.3f   Diff: %1.3f   ",
$db_hours, $hours, $hours - $db_hours;
            $bad_files++;
            printf "Bad duration (%s:%s:%s), need to fix %s\n", $hrs,
$mins, $secs, $f;
            system "/usr/bin/dd bs=1M skip=3 if=$f of=$f.fix";
            system "rm $f";
            system "mv $f.fix $f";
            print "Fixed, running mythcommflag\n";
            system "/usr/bin/mythcommflag -q --rebuild --file $f >>
/tmp/re-commflag.log 2>&1 &";
        }
        #else {
        #    printf "Database: %1.3f   ffprobe: %1.3f   Diff: %1.3f  
", $db_hours, $hours, $hours - $db_hours;
        #    printf "Valid duration (%s:%s:%s) found in %s\n", $hrs,
$mins, $secs, $f;
        #}
    }
}

print "Checked duration on $files_checked file";
print 's' if $files_checked != 1;
print ", found $bad_files that need fixing\n";

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org