Mailing List Archive: expire errors

expire errors

Aug 28, 2020, 7:34 AM

Post #1 of 5 (535 views)

I have recently moved to a new FE/BE combo system (MythTV v31 on Fedora
32). In the process, I have noticed a number of these errors from
mythbackend:

Aug 28 08:08:25 seveneves.gregandeva.net mythbackend[34639]: 2020-08-28
08:08:25.478068 N [34639/34695] Expire
autoexpire.cpp:633:SendDeleteMessages Expiring 9418 MB for 1858 at
2015-11-20T18:00:00Z => "Christmas Comes Home to Canaan"
Aug 28 08:08:25 seveneves.gregandeva.net mythbackend[34639]: 2020-08-28
08:08:25.481925 E [34639/34639] CoreContext
mainserver.cpp:3103:DoHandleDeleteRecording MainServer: ERROR when trying
to delete file: myth://mongoliad.gregandeva.net/1858_20151120173857.mpg.
File doesn't exist. Database metadata will not be removed

I checked, and ignoring the fact that "mongoliad" is the name of the old
backend machine, the recording file does not in fact exist. So the next
step is to run find_orphans.py (I just downloaded the latest version for
v31 and above, python3-based as is my Fedora 32 system). But it crashes as
follows:

[...long list of orphaned files...]]
mongoliad.gregandeva.net: The X-Files - Plus One
1655_20180118020000.ts
mongoliad.gregandeva.net: The X-Files - This
1655_20180111020000.ts

Count: 154
Are you sure you want to continue?
> yes
Traceback (most recent call last):
File "/usr/local/bin/find_orphans.py", line 230, in <module>
main()
File "/usr/local/bin/find_orphans.py", line 214, in main
opt[1](opt[2])
File "/usr/local/bin/find_orphans.py", line 129, in delete_recs
rec.delete(True, True)
File "/usr/lib/python3.8/site-packages/MythTV/dataheap.py", line 377, in
delete
return self.getProgram().delete(force, rerecord)
File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 975, in
delete
res = int(be.deleteRecording(self, force=force))
File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 671, in
deleteRecording
[command,program.toString()]))
File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 965, in
toString
return BACKEND_SEP.join(self._deprocess())
File "/usr/lib/python3.8/site-packages/MythTV/altdict.py", line 178, in
_deprocess
data[i] = self._inv_trans[self._field_type[i]](v)
File "/usr/lib/python3.8/site-packages/MythTV/altdict.py", line 113, in
<lambda>
lambda x: str(int(x.timestamp())),
File "/usr/lib/python3.8/site-packages/MythTV/utility/dt.py", line 481,
in timestamp
return ((utc_naive - utc_epoch).total_seconds())
TypeError: can't subtract offset-naive and offset-aware datetimes

Has anyone else seen this (and better yet, have a patch to fix it)?

How serious are the mythbackend errors? As far as I can see, the system is
working fine other than this, so it may be just cosmetic, but it does mean
at least that there is some cruft in my database that it would be nice to
get rid of.

As of now, I am nowhere near close to filling my recording storage
directories, but I am afraid that when I do start to get close, the expire
system may not actually be working correctly.

Thanks,
--Greg

Re: expire errors [ In reply to ]

rcrernst at gmail

Aug 29, 2020, 5:37 AM

Post #2 of 5 (527 views)

Permalink

On Fri, Aug 28, 2020 at 4:37 PM Greg Woods <greg@gregandeva.net> wrote:

> I have recently moved to a new FE/BE combo system (MythTV v31 on Fedora
> 32). In the process, I have noticed a number of these errors from
> mythbackend:
>
> Aug 28 08:08:25 seveneves.gregandeva.net mythbackend[34639]: 2020-08-28
> 08:08:25.478068 N [34639/34695] Expire
> autoexpire.cpp:633:SendDeleteMessages Expiring 9418 MB for 1858 at
> 2015-11-20T18:00:00Z => "Christmas Comes Home to Canaan"
> Aug 28 08:08:25 seveneves.gregandeva.net mythbackend[34639]: 2020-08-28
> 08:08:25.481925 E [34639/34639] CoreContext
> mainserver.cpp:3103:DoHandleDeleteRecording MainServer: ERROR when trying
> to delete file: myth://mongoliad.gregandeva.net/1858_20151120173857.mpg.
> File doesn't exist. Database metadata will not be removed
>
> I checked, and ignoring the fact that "mongoliad" is the name of the old
> backend machine, the recording file does not in fact exist. So the next
> step is to run find_orphans.py (I just downloaded the latest version for
> v31 and above, python3-based as is my Fedora 32 system). But it crashes as
> follows:
>
> [...long list of orphaned files...]]
> mongoliad.gregandeva.net: The X-Files - Plus One
> 1655_20180118020000.ts
> mongoliad.gregandeva.net: The X-Files - This
> 1655_20180111020000.ts
>
> Count: 154
> Are you sure you want to continue?
> > yes
> Traceback (most recent call last):
> File "/usr/local/bin/find_orphans.py", line 230, in <module>
> main()
> File "/usr/local/bin/find_orphans.py", line 214, in main
> opt[1](opt[2])
> File "/usr/local/bin/find_orphans.py", line 129, in delete_recs
> rec.delete(True, True)
> File "/usr/lib/python3.8/site-packages/MythTV/dataheap.py", line 377, in
> delete
> return self.getProgram().delete(force, rerecord)
> File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 975,
> in delete
> res = int(be.deleteRecording(self, force=force))
> File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 671,
> in deleteRecording
> [command,program.toString()]))
> File "/usr/lib/python3.8/site-packages/MythTV/mythproto.py", line 965,
> in toString
> return BACKEND_SEP.join(self._deprocess())
> File "/usr/lib/python3.8/site-packages/MythTV/altdict.py", line 178, in
> _deprocess
> data[i] = self._inv_trans[self._field_type[i]](v)
> File "/usr/lib/python3.8/site-packages/MythTV/altdict.py", line 113, in
> <lambda>
> lambda x: str(int(x.timestamp())),
> File "/usr/lib/python3.8/site-packages/MythTV/utility/dt.py", line 481,
> in timestamp
> return ((utc_naive - utc_epoch).total_seconds())
> TypeError: can't subtract offset-naive and offset-aware datetimes
>
> Has anyone else seen this (and better yet, have a patch to fix it)?
>
> How serious are the mythbackend errors? As far as I can see, the system is
> working fine other than this, so it may be just cosmetic, but it does mean
> at least that there is some cruft in my database that it would be nice to
> get rid of.
>
> As of now, I am nowhere near close to filling my recording storage
> directories, but I am afraid that when I do start to get close, the expire
> system may not actually be working correctly.
>
> Thanks,
> --Greg
>

For changing the hostname properly, see
https://www.mythtv.org/wiki/Database_Backup_and_Restore
section "Change the hostname of a MythTV frontend or backend"

For the utility "find_orphans.py", see the latest changes on
https://www.mythtv.org/wiki/Find_orphans.py
and the forum entry
https://forum.mythtv.org/viewtopic.php?f=6&t=3793
Note: The mentioned bug (1) was fixed in June 2020.
Could you please update your installed version of mythtv?

(1) #13622) <https://code.mythtv.org/trac/ticket/13622>

Roland

Re: expire errors [ In reply to ]

greg at gregandeva

Aug 29, 2020, 8:58 AM

Post #3 of 5 (526 views)

Permalink

On Sat, Aug 29, 2020 at 6:37 AM Roland Ernst <rcrernst@gmail.com> wrote:

>
>
> On Fri, Aug 28, 2020 at 4:37 PM Greg Woods <greg@gregandeva.net> wrote:
>
>>
>> For changing the hostname properly, see
> https://www.mythtv.org/wiki/Database_Backup_and_Restore
> section "Change the hostname of a MythTV frontend or backend"
>

Thanks for the pointer to that, I completely missed it and now it appears I
am hosed, because my system has been running with its new name for several
weeks now already and I have 81 recordings with the new host name and over
1000 with the old one. I will have to contemplate my options. One of them
is to just delete the 81 new recordings, but I'm willing to bet that isn't
as simple as just deleting the entries in the "recorded" table. I might be
able to accomplish that by restoring from a database backup made before the
host change and then doing the host name change procedure. I presume that
will create 81 new orphaned recordings, which I might even be able to
retrieve if I can find a way to get find_orphans.py working. Unfortunately
the crash is happening in one of the modules supplied with MythTV, so it's
probably not fixable by just hacking the find_orphans.py script.

>
> For the utility "find_orphans.py", see the latest changes on
> https://www.mythtv.org/wiki/Find_orphans.py
>

This is the version I am using, just downloaded yesterday because the
previous one I had was still the old Python 2 version, which of course
doesn't work at all any more.

> and the forum entry
> https://forum.mythtv.org/viewtopic.php?f=6&t=3793
>
Note: The mentioned bug (1) was fixed in June 2020.
> Could you please update your installed version of mythtv?
>

Not very easily. Right now I am using the version from RPM Fusion, which
is mythtv-31.0-3.20200527gitfc90482281.fc32.x86_64. We can see from the
date embedded in the version number that this predates the June 2020 fix.
I'll see if I can find out how to let the RPM Fusion folks know.

Re: expire errors [ In reply to ]

stephen_agent at jsw

Aug 29, 2020, 9:21 AM

Post #4 of 5 (526 views)

Permalink

On Sat, 29 Aug 2020 09:58:10 -0600, you wrote:

>On Sat, Aug 29, 2020 at 6:37 AM Roland Ernst <rcrernst@gmail.com> wrote:
>
>>
>>
>> On Fri, Aug 28, 2020 at 4:37 PM Greg Woods <greg@gregandeva.net> wrote:
>>
>>>
>>> For changing the hostname properly, see
>> https://www.mythtv.org/wiki/Database_Backup_and_Restore
>> section "Change the hostname of a MythTV frontend or backend"
>>
>
>Thanks for the pointer to that, I completely missed it and now it appears I
>am hosed, because my system has been running with its new name for several
>weeks now already and I have 81 recordings with the new host name and over
>1000 with the old one. I will have to contemplate my options. One of them
>is to just delete the 81 new recordings, but I'm willing to bet that isn't
>as simple as just deleting the entries in the "recorded" table. I might be
>able to accomplish that by restoring from a database backup made before the
>host change and then doing the host name change procedure. I presume that
>will create 81 new orphaned recordings, which I might even be able to
>retrieve if I can find a way to get find_orphans.py working.

One possible route to fixing your hostname problem would be to install
mythexport/mythimport and export all the new recordings. You could
then restore from the old backup and change the hostname, then use
mythimport to restore the new recordings. However, I have not yet
done a Python 3 version of mythimport. It should not be too difficult
to do as I am in the process of converting all my Python 2 to Python
3, so let me know if you want to try this.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: expire errors [ In reply to ]

greg at gregandeva

Aug 29, 2020, 9:59 AM

Post #5 of 5 (526 views)

Permalink

On Sat, Aug 29, 2020 at 10:22 AM Stephen Worthington <
stephen_agent@jsw.gen.nz> wrote:

>
> One possible route to fixing your hostname problem would be to install
> mythexport/mythimport and export all the new recordings. You could
> then restore from the old backup and change the hostname, then use
> mythimport to restore the new recordings.
>

I will add that to the list of options I need to consider. My problem is
that I have also made other changes to the database since the upgrade,
because I dropped cable service and went back to OTA only with Sling, then
decided there were serious limitations to Sling's DVR; primarily, I can't
watch a recording until it is completed, and Hulu shares the same issue so
it's not just one streaming service. This one is a show stopper for me
because one of my main uses of the DVR is to time-shift games so that I can
start watching an hour late and be able to skip or FF commercials. So now
I'm back with Comcast (mysteriously I am now getting a much better price
:-) So I have deleted and added channels and capture cards, as well as
making a few other tweaks (like upping the channel recpriority on some HD
channels that have SD equivalents). Redoing all this is a pretty big task
too, so "restoring from an old backup" is not quite that simple in my case.

--Greg