Mailing List Archive

Help - Possible Corrupted Database
I did something stupid which resulted in my root FS becoming 100% full
while mythtv was recording and I attempted to schedule a recording.  Now
when I use mythweb to look at upcoming recordings it shows nothing :-(.


I suspect that the database may have gotten corrupted when mythtv tried
to write to it while recording, etc.  What is the best approach to
getting things back to normal?

I have a database backup from 2:00am this morning, so that would be
close if needed.

Ooh.  I just checked again (upcoming recordings) and everything is
back.  I have a not so peaceful easy feeling about this.

Jay

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Help - Possible Corrupted Database [ In reply to ]
On Sat, 25 Mar 2023 13:10:50 -0700, you wrote:

>I did something stupid which resulted in my root FS becoming 100% full
>while mythtv was recording and I attempted to schedule a recording.? Now
>when I use mythweb to look at upcoming recordings it shows nothing :-(.
>
>
>I suspect that the database may have gotten corrupted when mythtv tried
>to write to it while recording, etc.? What is the best approach to
>getting things back to normal?
>
>I have a database backup from 2:00am this morning, so that would be
>close if needed.
>
>Ooh.? I just checked again (upcoming recordings) and everything is
>back.? I have a not so peaceful easy feeling about this.
>
>Jay

Shut down mythbackend. Clear enough free space for MySQL/MariaDB to
make a copy of your largest database table and its index files. To
find out what that is, do this command:

ls -alSh /var/lib/mysql/mythconverg/ | head -n 20

This will normally show the recordedseek table is by far the largest.
Here is mine:

root@mypvr:~# ls -alSh /var/lib/mysql/mythconverg/ | head -n 20
total 20G
-rw-rw---- 1 mysql mysql 11G Mar 26 16:04 recordedseek.MYD
-rw-rw---- 1 mysql mysql 9.1G Mar 26 16:04 recordedseek.MYI
-rw-rw---- 1 mysql mysql 24M Mar 26 16:04 oldrecorded.MYD
-rw-rw---- 1 mysql mysql 24M Mar 26 16:04 oldrecorded.MYI
-rw-rw---- 1 mysql mysql 22M Mar 26 10:33 music_streams.ibd
-rw-rw---- 1 mysql mysql 21M Mar 26 16:04 recorded.MYD
-rw-rw---- 1 mysql mysql 18M Mar 26 10:33 gallery_files.MYD
-rw-rw---- 1 mysql mysql 14M Mar 26 16:04 recordedprogram.MYD
-rw-rw---- 1 mysql mysql 14M Mar 26 16:04 recordedmarkup.MYI
-rw-rw---- 1 mysql mysql 13M Mar 26 16:04 recordedmarkup.MYD
-rw-rw---- 1 mysql mysql 12M Mar 26 10:46 videometadata.MYD
-rw-rw---- 1 mysql mysql 12M Mar 26 15:11 program.MYD
-rw-rw---- 1 mysql mysql 11M Mar 26 15:11 program.MYI
-rw-rw---- 1 mysql mysql 8.0M Mar 26 10:33 mythsgu_archived.ibd
-rw-rw---- 1 mysql mysql 7.0M Mar 26 10:33 record_tmp.ibd
-rw-rw---- 1 mysql mysql 5.7M Mar 26 16:04 recorded.MYI
-rw-rw---- 1 mysql mysql 4.8M Mar 26 16:04 recordedprogram.MYI
-rw-rw---- 1 mysql mysql 3.8M Mar 26 16:04 recordedfile.MYD
-rw-rw---- 1 mysql mysql 2.6M Mar 26 10:49 videometadata.MYI

So I need at least 20.1 Gibytes free to be able to repair the
database, and there needs to be a little more than that for other
things to work in the background, so say 22 Gibytes needs to be free.
I have 23 Gibytes free:

root@mypvr:~# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p4 122G 94G 23G 81% /

Then run the optimize_mythdb script that should be in your
/etc/cron.daily directory so it gets run automatically every day:

sudo /etc/cron.daily/optimize_mythdb

If it is not there, then there is a copy installed by the MythTV
packages:

sudo
/usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl

(That should be all one long line - my email client wraps long lines)

That should check and repair all the database tables. Usually, if a
recording was in progress when the space ran out, recordedseek will be
corrupt, but optimize_mythdb will repair it. And if you are lucky,
that will be the only table affected. If you have done any recordings
while it was corrupt, those recordings will not have had their seek
tables written to recordedseek and you will need to fix that by
running "mythcommflag --rebuild" on each of those recordings. If you
do commercial skip processing, you would also then need to redo that
as it is also stored in recorded seek - use mythcommflag with the
settings you use for commercial skip processing. Make sure to do the
--rebuild first, as a separate run of mythcommflag as they can not be
combined and --rebuild will delete the results of commercial skip
processing.

If other tables have been corrupted, then you may have permanent
damage there and have to restore from your backup. Run
optimize_mythdb again until no tables are repaired, then see if
everything is working before deciding to do a database restore.

If you do restore the database, then any recordings done since the
backup will become orphaned - find the recording files
(find_orphans.py) and move them to a videos directory, then rescan
your videos:

mythutil --scanvideos
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Help - Possible Corrupted Database [ In reply to ]
On 3/25/23 22:37, Stephen Worthington wrote:
>
> sudo
> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
>
> (That should be all one long line - my email client wraps long lines)
>
> That should check and repair all the database tables. Usually, if a
> recording was in progress when the space ran out, recordedseek will be
> corrupt, but optimize_mythdb will repair it. And if you are lucky,
> that will be the only table affected. If you have done any recordings
> while it was corrupt, those recordings will not have had their seek
> tables written to recordedseek and you will need to fix that by
> running "mythcommflag --rebuild" on each of those recordings. If you
> do commercial skip processing, you would also then need to redo that
> as it is also stored in recorded seek - use mythcommflag with the
> settings you use for commercial skip processing. Make sure to do the
> --rebuild first, as a separate run of mythcommflag as they can not be
> combined and --rebuild will delete the results of commercial skip
> processing.
>
> If other tables have been corrupted, then you may have permanent
> damage there and have to restore from your backup. Run
> optimize_mythdb again until no tables are repaired, then see if
> everything is working before deciding to do a database restore.
>
> If you do restore the database, then any recordings done since the
> backup will become orphaned - find the recording files
> (find_orphans.py) and move them to a videos directory, then rescan
> your videos:
>
> mythutil --scanvideos
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums:https://forum.mythtv.org

Thanks for the guidance. Like I said, shutdown (on the new machine with
old system kernel copied over) was working just  fine until I added sas
9211 hba card and a couple disks that were on my old system. Now it
completes every step and comes to reboot target and then reports that
some one is trying to do some extra work that kernel is ignoring. I will
get the exact message and post it here. I am almost sure that if I take
out the hba card, everything will work fine. However, I do not have
enough SATA ports that I need this card. Apparently, this is one of the
cards that has excellent (long time) support on the linux side. Since my
backend is always on machine, I am not too inconvenienced by this
limitation as of now. Still it bothers me that I have to push the power
button and hold it to force the situation.

  I will try to post on one of the kernel threads or debian-user to see
what I get. I doubt there are any filesys corruption issues because on
screen log shows everything except actual power off or reboot is done.
So, I am less worried about pushing the power button.  However, I will
try all of your above suggestions and will learn what internal system
state is and go from there.

Regards
Ramesh
Re: Help - Possible Corrupted Database [ In reply to ]
On 3/26/23 11:19, Ram Ramesh wrote:
> On 3/25/23 22:37, Stephen Worthington wrote:
>>
>> sudo
>> /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl
>>
>> (That should be all one long line - my email client wraps long lines)
>>
>> That should check and repair all the database tables. Usually, if a
>> recording was in progress when the space ran out, recordedseek will be
>> corrupt, but optimize_mythdb will repair it. And if you are lucky,
>> that will be the only table affected. If you have done any recordings
>> while it was corrupt, those recordings will not have had their seek
>> tables written to recordedseek and you will need to fix that by
>> running "mythcommflag --rebuild" on each of those recordings. If you
>> do commercial skip processing, you would also then need to redo that
>> as it is also stored in recorded seek - use mythcommflag with the
>> settings you use for commercial skip processing. Make sure to do the
>> --rebuild first, as a separate run of mythcommflag as they can not be
>> combined and --rebuild will delete the results of commercial skip
>> processing.
>>
>> If other tables have been corrupted, then you may have permanent
>> damage there and have to restore from your backup. Run
>> optimize_mythdb again until no tables are repaired, then see if
>> everything is working before deciding to do a database restore.
>>
>> If you do restore the database, then any recordings done since the
>> backup will become orphaned - find the recording files
>> (find_orphans.py) and move them to a videos directory, then rescan
>> your videos:
>>
>> mythutil --scanvideos
>> _______________________________________________
>> mythtv-users mailing list
>> mythtv-users@mythtv.org
>> http://lists.mythtv.org/mailman/listinfo/mythtv-users
>> http://wiki.mythtv.org/Mailing_List_etiquette
>> MythTV Forums:https://forum.mythtv.org
>
> Thanks for the guidance. Like I said, shutdown (on the new machine
> with old system kernel copied over) was working just  fine until I
> added sas 9211 hba card and a couple disks that were on my old system.
> Now it completes every step and comes to reboot target and then
> reports that some one is trying to do some extra work that kernel is
> ignoring. I will get the exact message and post it here. I am almost
> sure that if I take out the hba card, everything will work fine.
> However, I do not have enough SATA ports that I need this card.
> Apparently, this is one of the cards that has excellent (long time)
> support on the linux side. Since my backend is always on machine, I am
> not too inconvenienced by this limitation as of now. Still it bothers
> me that I have to push the power button and hold it to force the
> situation.
>
>   I will try to post on one of the kernel threads or debian-user to
> see what I get. I doubt there are any filesys corruption issues
> because on screen log shows everything except actual power off or
> reboot is done. So, I am less worried about pushing the power button. 
> However, I will try all of your above suggestions and will learn what
> internal system state is and go from there.
>
> Regards
> Ramesh
>

Sorry, please ignore. I posted on the wrong thread.

Regards
Ramesh