Mailing List Archive

MythTV jobs getting stuck
Not near my system right now but looked at the status page online and
noticed a long list of queued jobs. (System currently has two cores and I
limit the number of running jobs to 1)

The box was stuck on a mythmetalookup job for a show that aired last night.
Occasionally I will see it get stuck one a commflag job. Maybe it happens a
couple of times a month. I don't know if this is a feature request or
something that could be handled by the OS, but is it possible to check and
see if certain MythTV processes have been running for a ridiculously long
time and just terminate them?
Re: MythTV jobs getting stuck [ In reply to ]
On 6/6/20 6:42 PM, Ian Evans wrote:
> Not near my system right now but looked at the status page online and
> noticed a long list of queued jobs. (System currently has two cores
> and I limit the number of running jobs to 1)
>
> The box was stuck on a mythmetalookup job for a show that aired last
> night. Occasionally I will see it get stuck one a commflag job. Maybe
> it happens a couple of times a month. I don't know if this is a
> feature request or something that could be handled by the OS, but is
> it possible to check and see if certain MythTV processes have been
> running for a ridiculously long time and just terminate them?
>
There may be a bug here - I suspect if it does not get a response to a
request to a web site that it waits for a very long time. Probably some
better checking in the metadata lookup job would help.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: MythTV jobs getting stuck [ In reply to ]
On 6/6/20 6:35 PM, Peter Bennett wrote:
>
> On 6/6/20 6:42 PM, Ian Evans wrote:
>> Not near my system right now but looked at the status page online and
>> noticed a long list of queued jobs. (System currently has two cores
>> and I limit the number of running jobs to 1)
>>
>> The box was stuck on a mythmetalookup job for a show that aired last
>> night. Occasionally I will see it get stuck one a commflag job. Maybe
>> it happens a couple of times a month. I don't know if this is a
>> feature request or something that could be handled by the OS, but is
>> it possible to check and see if certain MythTV processes have been
>> running for a ridiculously long time and just terminate them?
>>
> There may be a bug here - I suspect if it does not get a response to a
> request to a web site that it waits for a very long time. Probably
> some better checking in the metadata lookup job would help.

Getting stuck on metalookup happens a few times a month for me, using
Ubuntu 16.04 / MythTV 29.  It had been happening for years, but I blamed
it in part from being on a satellite Internet connection.  However, I'm
now on terrestrial Internet and still see the same problem.  I've never
seen anything in the logs to hint at the problem.  Restarting
mythtv-backend 'fixes' the job queue.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: MythTV jobs getting stuck [ In reply to ]
On 6/6/20 5:13 PM, Martin Compton wrote:
> On 6/6/20 6:35 PM, Peter Bennett wrote:
>>
>> On 6/6/20 6:42 PM, Ian Evans wrote:
>>> Not near my system right now but looked at the status page online
>>> and noticed a long list of queued jobs. (System currently has two
>>> cores and I limit the number of running jobs to 1)
>>>
>>> The box was stuck on a mythmetalookup job for a show that aired last
>>> night. Occasionally I will see it get stuck one a commflag job.
>>> Maybe it happens a couple of times a month. I don't know if this is
>>> a feature request or something that could be handled by the OS, but
>>> is it possible to check and see if certain MythTV processes have
>>> been running for a ridiculously long time and just terminate them?
>>>
>> There may be a bug here - I suspect if it does not get a response to
>> a request to a web site that it waits for a very long time. Probably
>> some better checking in the metadata lookup job would help.
>
> Getting stuck on metalookup happens a few times a month for me, using
> Ubuntu 16.04 / MythTV 29.  It had been happening for years, but I
> blamed it in part from being on a satellite Internet connection. 
> However, I'm now on terrestrial Internet and still see the same
> problem.  I've never seen anything in the logs to hint at the
> problem.  Restarting mythtv-backend 'fixes' the job queue.

The only problem I've had is mythcommflag running forever.  It happens a
lot on Evening Magazine, a 1/2 hour show shown in Seattle at 7:30 PM,
but not always.  It also happens on any recording with the least
corruption (which may be a common problem with Evening Magazine).  Once
it happens, the jobs just queue up behind it, so I wrote a perl script
that cron runs at 2:00 AM and 10:00 AM.  By the time I look, there are
no queued mythcommflag processes any more.

Dave D.

Here's the script:

#!/usr/bin/perl -l

use strict;

my @comflags = `/usr/bin/ps xwa -o pid,etime,command | grep mythcommflag
| grep -v grep`;
chomp @comflags;
#  The ps command, above, results in something like this for a normal
process.  A hung process has an etime (2nd field) of something like
7-18:42:20 (7 days, 18 hours!)
# 1009821       04:04 /usr/bin/mythcommflag -j 12276 --noprogress
--verbose general --logpath /var/log/mythtv --loglevel info

foreach my $p (@comflags) {
        $p =~ s/^\s+|\s+$//;  # strip leading/trailing white space
        my ($pid, $etime, $rest) = split /\s+/, $p, 3;

        my @time_parts = split /\D+/, $etime;

        # More than a day?  Add the days to hours
        if (scalar @time_parts > 3) {
                my $days = shift @time_parts;
                $time_parts[0] += 24 * $days;
        }

        # more than 3 hours?
        if ($time_parts[0] > 2) {
                print "PID $pid has elapsed time of $etime
($time_parts[0]+ hours) on \"$p\"\n";
                print "Killing process $pid\n";
                system "kill -9 $pid";

                # "next" (instead of "exit") would check them all.
Hopefully the first one is the
                # culprit and this will free up the system to do the others
                exit;
        }
}

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: MythTV jobs getting stuck [ In reply to ]
On Sun, Jun 7, 2020, 1:47 PM DaveD, <mythtv@guiplot.com> wrote:

> On 6/6/20 5:13 PM, Martin Compton wrote:
> > On 6/6/20 6:35 PM, Peter Bennett wrote:
> >>
> >> On 6/6/20 6:42 PM, Ian Evans wrote:
> >>> Not near my system right now but looked at the status page online
> >>> and noticed a long list of queued jobs. (System currently has two
> >>> cores and I limit the number of running jobs to 1)
> >>>
> >>> The box was stuck on a mythmetalookup job for a show that aired last
> >>> night. Occasionally I will see it get stuck one a commflag job.
> >>> Maybe it happens a couple of times a month. I don't know if this is
> >>> a feature request or something that could be handled by the OS, but
> >>> is it possible to check and see if certain MythTV processes have
> >>> been running for a ridiculously long time and just terminate them?
> >>>
> >> There may be a bug here - I suspect if it does not get a response to
> >> a request to a web site that it waits for a very long time. Probably
> >> some better checking in the metadata lookup job would help.
> >
> > Getting stuck on metalookup happens a few times a month for me, using
> > Ubuntu 16.04 / MythTV 29. It had been happening for years, but I
> > blamed it in part from being on a satellite Internet connection.
> > However, I'm now on terrestrial Internet and still see the same
> > problem. I've never seen anything in the logs to hint at the
> > problem. Restarting mythtv-backend 'fixes' the job queue.
>
> The only problem I've had is mythcommflag running forever. It happens a
> lot on Evening Magazine, a 1/2 hour show shown in Seattle at 7:30 PM,
> but not always. It also happens on any recording with the least
> corruption (which may be a common problem with Evening Magazine). Once
> it happens, the jobs just queue up behind it, so I wrote a perl script
> that cron runs at 2:00 AM and 10:00 AM. By the time I look, there are
> no queued mythcommflag processes any more.
>
> Dave D.
>
> Here's the script:
>
> #!/usr/bin/perl -l
>
> use strict;
>
> my @comflags = `/usr/bin/ps xwa -o pid,etime,command | grep mythcommflag
> | grep -v grep`;
> chomp @comflags;
> # The ps command, above, results in something like this for a normal
> process. A hung process has an etime (2nd field) of something like
> 7-18:42:20 (7 days, 18 hours!)
> # 1009821 04:04 /usr/bin/mythcommflag -j 12276 --noprogress
> --verbose general --logpath /var/log/mythtv --loglevel info
>
> foreach my $p (@comflags) {
> $p =~ s/^\s+|\s+$//; # strip leading/trailing white space
> my ($pid, $etime, $rest) = split /\s+/, $p, 3;
>
> my @time_parts = split /\D+/, $etime;
>
> # More than a day? Add the days to hours
> if (scalar @time_parts > 3) {
> my $days = shift @time_parts;
> $time_parts[0] += 24 * $days;
> }
>
> # more than 3 hours?
> if ($time_parts[0] > 2) {
> print "PID $pid has elapsed time of $etime
> ($time_parts[0]+ hours) on \"$p\"\n";
> print "Killing process $pid\n";
> system "kill -9 $pid";
>
> # "next" (instead of "exit") would check them all.
> Hopefully the first one is the
> # culprit and this will free up the system to do the
> others
> exit;
> }
> }
>
>
>
As Chong once said to Cheech: "Oh cool beans, man."

That script seems to be just what the doctor ordered.

>
>