Mailing List Archive

-----Original message-----
From: Jay Harbeston <jharbestonus@gmail.com>
Sent: Tuesday 6th June 2023 17:32
To: Discussion about MythTV <mythtv-users@mythtv.org>
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings



On Jun 6, 2023, at 12:22 PM, James Abernathy <jfabernathy@gmail.com> wrote:


Database changed
MariaDB [mythconverg]> SHOW VARIABLES LIKE "secure_file_priv";
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| secure_file_priv |       |
+------------------+-------+
1 row in set (0.001 sec)

Looks empty to me.


I was afraid of that. My attempts to find a solution in mysql failed to find a solution. So much security so little time! :-)

Jay



This has worked for me in mysql, I presume it would also work for MariaDB:


grant file on *.* to 'mythtv'@'localhost';



The problem then was that the output file was owned by the mysql user, so I had to add the mysql group to the user that I was using to run ffcut.sh



Regards,

Dave.

I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it.

I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version.

With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.

It will require python version 3+.

When I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.

Jay
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>
>
> I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it.
>
> I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version.
>
> With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.
>
> It will require python version 3+.
>
> jamesWhen I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.

Can you cut h264 recordings?

I have 2 trancode-lossless scripts, 1 for each

James
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

> On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:
>
>
>
>> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com <mailto:jharbestonus@gmail.com>> wrote:
>>
>>
>>
>> I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it.
>>
>> I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version.
>>
>> With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.
>>
>> It will require python version 3+.
>>
>> jamesWhen I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.
>
> Can you cut h264 recordings?
>
> I have 2 trancode-lossless scripts, 1 for each
>

I know it can cut mp4 ts streams…

If ffmpeg can cut h264 and there is a list of cuts in mythtv, it should be able to. I do know that mp4s created from handbrake can not create a seektaable, so cannot be flagged in mythtv.

Hi Jay,



I had to make a few modifications to your ffcut.sh script to get it to work - modifying the mysql commands, slightly as they didn't work on my system as written, but I soon resolved that.



The other modifications I had to make were to the ffmpeg commands, since my h264 recordings with AC3 5.1 audio were transcoding without audio - I changed the commands as follows:



 cmdffmpeg="ffmpeg -i $infile -t $duration -ss $ss -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"



and



cmdffmpeg="ffmpeg -f concat -safe 0 -i $tmpdir/filelist.txt -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"



this resolved the audio problems, but I am unable to rebuild the seektable for the newly transcoded recordings





I imagine that the ffmpeg command needs some further modification, but I have no idea what that would be - maybe this will help you, though, with your python script?



Regards,

Dave 





-----Original message-----
From: Jay Harbeston <jharbestonus@gmail.com>
Sent: Thursday 8th June 2023 0:20
To: Discussion about MythTV <mythtv-users@mythtv.org>
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings



On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:



On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com <mailto:jharbestonus@gmail.com> > wrote:



I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it. 

I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version. 

With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.

It will require python version 3+.

jamesWhen I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.

Can you cut h264 recordings?

I have 2 trancode-lossless scripts, 1 for each


I know it can cut mp4 ts streams… 

If  ffmpeg can cut h264 and there is a list of cuts in mythtv, it should be able to.  I do know that mp4s created from handbrake can not create a seektaable, so cannot be flagged in mythtv.

On Wed, Jun 7, 2023 at 7:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:

>
>
> On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:
>
>
>
> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>
>
> I have created a python version of ffcut.sh and am currently making sure I
> have most of the kinks out of it.
>
> I’m thinking tomorrow I should be able to post the python version the same
> way as I did the shell version.
>
> With the python version I did not have to create the text files as
> intermediary holding arrays for data that needed to be used.
>
> It will require python version 3+.
>
> jamesWhen I created it originally, mythtranscode would not handle mp4
> streams that we were getting from the HDHRs when silicon dust did the 4 or
> so month experiment of being a content provider from the cloud. The ffmpeg
> version ffcut.sh and soon the ffcut.py version will be able to handle both
> mp4 and mpeg 2 transport streams.
>
>
I'd be interested in looking at the python script also.

BTW, I did get ffcut working for me to a point. I have to may the mysql
commands look like this if password was 1234
sudo mysql -u root -p1234

That works from the console, but not sure what will happen in a UserJob
when it encounters a sudo?

Jim A

On 08/06/2023 12:45, James Abernathy wrote:
> On Wed, Jun 7, 2023 at 7:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>>
>>
>> On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:
>>
>>
>>
>> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com> wrote:
>>
>>
>>
>> I have created a python version of ffcut.sh and am currently making sure I
>> have most of the kinks out of it.
>>
>> I’m thinking tomorrow I should be able to post the python version the same
>> way as I did the shell version.
>>
>> With the python version I did not have to create the text files as
>> intermediary holding arrays for data that needed to be used.
>>
>> It will require python version 3+.
>>
>> jamesWhen I created it originally, mythtranscode would not handle mp4
>> streams that we were getting from the HDHRs when silicon dust did the 4 or
>> so month experiment of being a content provider from the cloud. The ffmpeg
>> version ffcut.sh and soon the ffcut.py version will be able to handle both
>> mp4 and mpeg 2 transport streams.
>>
>>
> I'd be interested in looking at the python script also.
>
> BTW, I did get ffcut working for me to a point. I have to may the mysql
> commands look like this if password was 1234
> sudo mysql -u root -p1234
>
> That works from the console, but not sure what will happen in a UserJob
> when it encounters a sudo?
>
As I recall you can put something in the sudoers file for specific jobs and/or specific users, such
that you wouldn't require a password for that combination. If that wasn't so, half of unix wouldn't run.

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

-----Original message-----
From: Mike Perkins <mikep@randomtraveller.org.uk>
Sent: Thursday 8th June 2023 12:52
To: mythtv-users@mythtv.org
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings


On 08/06/2023 12:45, James Abernathy wrote:
> On Wed, Jun 7, 2023 at 7:21 PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>>
>>
>> On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:
>>
>>
>>
>> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com> wrote:
>>
>>
>>
>> I have created a python version of ffcut.sh and am currently making sure I
>> have most of the kinks out of it.
>>
>> I'm thinking tomorrow I should be able to post the python version the same
>> way as I did the shell version.
>>
>> With the python version I did not have to create the text files as
>> intermediary holding arrays for data that needed to be used.
>>
>> It will require python version 3+.
>>
>> jamesWhen I created it originally, mythtranscode would not handle mp4
>> streams that we were getting from the HDHRs when silicon dust did the 4 or
>> so month experiment of being a content provider from the cloud. The ffmpeg
>> version ffcut.sh and soon the ffcut.py version will be able to handle both
>> mp4 and mpeg 2 transport streams.
>>
>>
> I'd be interested in looking at the python script also.
>
> BTW, I did get ffcut working for me to a point. I have to may the mysql
> commands look like this if password was 1234
> sudo mysql -u root -p1234
>
> That works from the console, but not sure what will happen in a UserJob
> when it encounters a sudo?
>
As I recall you can put something in the sudoers file for specific jobs and/or specific users, such
that you wouldn't require a password for that combination. If that wasn't so, half of unix wouldn't run.

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org



Mike,



I modified Jay's script so that it uses the /etc/mythtv/config.xml file to get the user credentials:



confxml="/etc/mythtv/config.xml"

hostname=`grep "<Host" $confxml | sed -e "s;</.*>;;" | sed -e "s;<.*>;;" | sed -e "s/  *//g"`

username=`grep "UserName" $confxml | sed -e "s;</.*>;;" | sed -e "s;<.*>;;" | sed -e "s/  *//g"`

passwd=`grep "Password" $confxml | sed -e "s;</.*>;;" | sed -e "s;<.*>;;" | sed -e "s/  *//g"`

dbname=`grep "DatabaseName" $confxml | sed -e "s;</.*>;;" | sed -e "s;<.*>;;" | sed -e "s/  *//g"`



usercreds="-h$hostname -u$username -p$passwd $dbname"

echo "User Credentials = "$usercreds





Then I modified the sql query lines to use this info, such that the first query is:


echo "mythdir=$mythdir chanid=$chanid starttimeutc=$starttimeutc "

query0="select basename from recorded where chanid = $chanid and starttime = '$starttimeutc';"



echo "query = $query0"



basename=`echo "$query0;" | mysql -N $usercreds`



echo "Basename = $basename"



By modifying it in this way, I do not need to use sudo, so long as I enable file privileges for the mysql user, as per an earlier email from me.



Obviously, I should really stop echoing the $usercreds variable :-) 



Regards,

Dave

> On Jun 8, 2023, at 6:34 AM, Lists <lists@the-pearsons.co.uk> wrote:
>
> Hi Jay,
>
> I had to make a few modifications to your ffcut.sh script to get it to work - modifying the mysql commands, slightly as they didn't work on my system as written, but I soon resolved that.
>
> The other modifications I had to make were to the ffmpeg commands, since my h264 recordings with AC3 5.1 audio were transcoding without audio - I changed the commands as follows:
>
> cmdffmpeg="ffmpeg -i $infile -t $duration -ss $ss -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"
>
> and
>
> cmdffmpeg="ffmpeg -f concat -safe 0 -i $tmpdir/filelist.txt -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"
>
> this resolved the audio problems, but I am unable to rebuild the seektable for the newly transcoded recordings
>
>

Interesting. I’ll try that on some of the other files I have. I would have that that -c copy would copy all audio streams as well as the video stream.

It has worked so far on my limited file tests I am using.


> I imagine that the ffmpeg command needs some further modification, but I have no idea what that would be - maybe this will help you, though, with your python script?
>

If it is an issue, it should as I am just using the python subprocess module to exec the cmdffmpeg string variable.

Jay

On Thu, Jun 8, 2023 at 7:53?AM Mike Perkins <mikep@randomtraveller.org.uk>
wrote:

> On 08/06/2023 12:45, James Abernathy wrote:
> > On Wed, Jun 7, 2023 at 7:21?PM Jay Harbeston <jharbestonus@gmail.com>
> wrote:
> >
> >>
> >>
> >> On Jun 7, 2023, at 7:13 PM, James <jam@tigger.ws> wrote:
> >>
> >>
> >>
> >> On 8 Jun 2023, at 7:08 am, Jay Harbeston <jharbestonus@gmail.com>
> wrote:
> >>
> >>
> >>
> >> I have created a python version of ffcut.sh and am currently making
> sure I
> >> have most of the kinks out of it.
> >>
> >> I’m thinking tomorrow I should be able to post the python version the
> same
> >> way as I did the shell version.
> >>
> >> With the python version I did not have to create the text files as
> >> intermediary holding arrays for data that needed to be used.
> >>
> >> It will require python version 3+.
> >>
> >> jamesWhen I created it originally, mythtranscode would not handle mp4
> >> streams that we were getting from the HDHRs when silicon dust did the 4
> or
> >> so month experiment of being a content provider from the cloud. The
> ffmpeg
> >> version ffcut.sh and soon the ffcut.py version will be able to handle
> both
> >> mp4 and mpeg 2 transport streams.
> >>
> >>
> > I'd be interested in looking at the python script also.
> >
> > BTW, I did get ffcut working for me to a point. I have to may the mysql
> > commands look like this if password was 1234
> > sudo mysql -u root -p1234
> >
> > That works from the console, but not sure what will happen in a UserJob
> > when it encounters a sudo?
> >
> As I recall you can put something in the sudoers file for specific jobs
> and/or specific users, such
> that you wouldn't require a password for that combination. If that wasn't
> so, half of unix wouldn't run.
>
> --
>
> Mike Perkins
>
>
You are correct. I added this line to a /etc/sudoer.d/mysql file
mythtv ALL = NOPASSWD: /usr/bin/mysql

Jim A

Dave,

> On Jun 8, 2023, at 6:34 AM, Lists <lists@the-pearsons.co.uk> wrote:
>
> Hi Jay,
>
> I had to make a few modifications to your ffcut.sh script to get it to work - modifying the mysql commands, slightly as they didn't work on my system as written, but I soon resolved that.
>
> The other modifications I had to make were to the ffmpeg commands, since my h264 recordings with AC3 5.1 audio were transcoding without audio - I changed the commands as follows:
>
> cmdffmpeg="ffmpeg -i $infile -t $duration -ss $ss -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"
>
> and
>
> cmdffmpeg="ffmpeg -f concat -safe 0 -i $tmpdir/filelist.txt -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"
>
> this resolved the audio problems, but I am unable to rebuild the seektable for the newly transcoded recordings
>


Can you do the following command on the file that has the audio go away? :

ffprobe -i file.ts


And post the results in a reply. I want to see how the audio streams get defined by ffmpeg.

Jay

-----Original message-----
From: Jay Harbeston <jharbestonus@gmail.com>
Sent: Thursday 8th June 2023 15:38
To: Discussion about MythTV <mythtv-users@mythtv.org>
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings

Dave,

On Jun 8, 2023, at 6:34 AM, Lists <lists@the-pearsons.co.uk> wrote:

Hi Jay,

I had to make a few modifications to your ffcut.sh script to get it to work - modifying the mysql commands, slightly as they didn't work on my system as written, but I soon resolved that.

The other modifications I had to make were to the ffmpeg commands, since my h264 recordings with AC3 5.1 audio were transcoding without audio - I changed the commands as follows:

 cmdffmpeg="ffmpeg -i $infile -t $duration -ss $ss -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"

and

cmdffmpeg="ffmpeg -f concat -safe 0 -i $tmpdir/filelist.txt -map 0:v -c:v copy -map 0:1 -c:a ac3 -ar 48000 $outfile"

this resolved the audio problems, but I am unable to rebuild the seektable for the newly transcoded recordings



Can you do the following command on the file that has the audio go away? : 

ffprobe -i  file.ts


And post the results in a reply. I want to see how the audio streams get defined by ffmpeg.

Jay



Sadly I don't have any of those files available now, but when I was checking them the other day, there was no audio track reported by mediainfo or vlc player. When I was watching the encoding, I could see that audio was reported as zero bytes



Regards,

Dave.

>
> Can you do the following command on the file that has the audio go away? :
>
> ffprobe -i file.ts
>
>
> And post the results in a reply. I want to see how the audio streams get defined by ffmpeg.
>
> Jay
>
> Sadly I don't have any of those files available now, but when I was checking them the other day, there was no audio track reported by mediainfo or vlc player. When I was watching the encoding, I could see that audio was reported as zero bytes

There may be a copy of the original file saved as file.ts.orig in your recordings directory if you didn’t add the the dontremove flag on the ffcut command .

Jay

On 6/7/2023 7:08 PM, Jay Harbeston wrote:
>
> I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it.
>
> I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version.
>
> With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.
>
> It will require python version 3+.
>
> When I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.
>
> Jay

One suggestion is to replace ffmpeg with mythffmpeg, since it is
included in the mythtv-common package and is essentially a renamed
ffmpeg. At least mythffmpeg is part of v32 mythtv-common. I assume it is
still packaged in later versions as well.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Wed, Jun 7, 2023 at 7:10?PM Jay Harbeston <jharbestonus@gmail.com> wrote:

>
>
> I have created a python version of ffcut.sh and am currently making sure I
> have most of the kinks out of it.
>
> I’m thinking tomorrow I should be able to post the python version the same
> way as I did the shell version.
>
> With the python version I did not have to create the text files as
> intermediary holding arrays for data that needed to be used.
>
> It will require python version 3+.
>
> When I created it originally, mythtranscode would not handle mp4 streams
> that we were getting from the HDHRs when silicon dust did the 4 or so month
> experiment of being a content provider from the cloud. The ffmpeg version
> ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and
> mpeg 2 transport streams.
>
> Jay
>

I got the ffcut.sh working with mariadb queued from the mythfrontend. and
it seemed to work correctly.
What I figured out is Not only do you have to run commflag but you need to
go into edit mode in mythfrontend and load the detected commercials and
edit them if needed and then save them.

Once the user job was completed what I saw in mythfrontend was all the
metadata still in the library listing but when I played the TV show, it was
22 minutes shorter, missing the commercials. Yey.
When checking the /srv/mythtv/recordings directory I had the original .ts
file copied to .ts.orig.

So if that's what it's supposed to do, it work.

Thanks

Jim A

Jim,

> On Jun 8, 2023, at 12:29 PM, James Abernathy <jfabernathy@gmail.com> wrote:
>
>
>
> On Wed, Jun 7, 2023 at 7:10?PM Jay Harbeston <jharbestonus@gmail.com <mailto:jharbestonus@gmail.com>> wrote:
>>
>>
>> I have created a python version of ffcut.sh and am currently making sure I have most of the kinks out of it.
>>
>> I’m thinking tomorrow I should be able to post the python version the same way as I did the shell version.
>>
>> With the python version I did not have to create the text files as intermediary holding arrays for data that needed to be used.
>>
>> It will require python version 3+.
>>
>> When I created it originally, mythtranscode would not handle mp4 streams that we were getting from the HDHRs when silicon dust did the 4 or so month experiment of being a content provider from the cloud. The ffmpeg version ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and mpeg 2 transport streams.
>>
>> Jay
>
> I got the ffcut.sh working with mariadb queued from the mythfrontend. and it seemed to work correctly.
> What I figured out is Not only do you have to run commflag but you need to go into edit mode in mythfrontend and load the detected commercials and edit them if needed and then save them.
>
> Once the user job was completed what I saw in mythfrontend was all the metadata still in the library listing but when I played the TV show, it was 22 minutes shorter, missing the commercials. Yey.
> When checking the /srv/mythtv/recordings directory I had the original .ts file copied to .ts.orig.
>
> So if that's what it's supposed to do, it work.
>

That is correct. In the python version that I am finishing up, there will be a remove option on the command line in mythtv settings.

It will be the final parameter, and if the 4th parameter is YES, the orig backup file WILL NOT be created. If it is NO, then the orig file WILL be created.


As per another comment, I am replacing ffmpeg with mythffmpeg as well.

The file that you processed, did it have 5.1 audio? If so, I am guessing that it came through correctly after ffcut?


Jay

I use https://github.com/angelaschmid/lossless_cut for saving in mythvideo

I recently updated it to python-3 locally. What I want to do is to tear out the closed caption manipulation (extracting and merging), as copying with ffmpeg does include close captions by default now.





> On 6/7/2023 7:08 PM, Jay Harbeston wrote:

>One suggestion is to replace ffmpeg with mythffmpeg



What I noticed is that one of the two (don’t remember which one) had a problem with closed captions when the first cut is not from the beginning.

I use https://github.com/angelaschmid/lossless_cut for saving in mythvideo

I recently updated it to python-3 locally. What I want to do is to tear out the closed caption manipulation (extracting and merging), as copying with mkvmerge (it is mkvmerge instead of ffmpeg, sorry) does include close captions by default now.





> On 6/7/2023 7:08 PM, Jay Harbeston wrote:

>One suggestion is to replace ffmpeg with mythffmpeg



What I noticed is that one of the two (don’t remember which one) had a problem with closed captions when the first cut is not from the beginning.

On Thu, Jun 8, 2023 at 12:59?PM Jay Harbeston <jharbestonus@gmail.com>
wrote:

> Jim,
>
> On Jun 8, 2023, at 12:29 PM, James Abernathy <jfabernathy@gmail.com>
> wrote:
>
>
>
> On Wed, Jun 7, 2023 at 7:10?PM Jay Harbeston <jharbestonus@gmail.com>
> wrote:
>
>>
>>
>> I have created a python version of ffcut.sh and am currently making sure
>> I have most of the kinks out of it.
>>
>> I’m thinking tomorrow I should be able to post the python version the
>> same way as I did the shell version.
>>
>> With the python version I did not have to create the text files as
>> intermediary holding arrays for data that needed to be used.
>>
>> It will require python version 3+.
>>
>> When I created it originally, mythtranscode would not handle mp4 streams
>> that we were getting from the HDHRs when silicon dust did the 4 or so month
>> experiment of being a content provider from the cloud. The ffmpeg version
>> ffcut.sh and soon the ffcut.py version will be able to handle both mp4 and
>> mpeg 2 transport streams.
>>
>> Jay
>>
>
> I got the ffcut.sh working with mariadb queued from the mythfrontend.
> and it seemed to work correctly.
> What I figured out is Not only do you have to run commflag but you need to
> go into edit mode in mythfrontend and load the detected commercials and
> edit them if needed and then save them.
>
> Once the user job was completed what I saw in mythfrontend was all the
> metadata still in the library listing but when I played the TV show, it was
> 22 minutes shorter, missing the commercials. Yey.
> When checking the /srv/mythtv/recordings directory I had the original .ts
> file copied to .ts.orig.
>
> So if that's what it's supposed to do, it work.
>
>
> That is correct. In the python version that I am finishing up, there will
> be a remove option on the command line in mythtv settings.
>
> It will be the final parameter, and if the 4th parameter is YES, the orig
> backup file WILL NOT be created. If it is NO, then the orig file WILL be
> created.
>
>
> As per another comment, I am replacing ffmpeg with mythffmpeg as well.
>
> The file that you processed, did it have 5.1 audio? If so, I am guessing
> that it came through correctly after ffcut?
>
>
> Jay
>

This particular video was a daytime game show so might not be 5.1, but the
stats on the audio are A52 (aka ac3 A52)

It's the same spec on the original and output of ffcut. I'll look for
something 5.1 tonight during primetime.

Jim A

Here is the first stab at the python version. It mimics the operation of the ffcut.sh, without all the db issues earlier mentioned, and uses the MythDB connection simplifying connection to the DB. I did change it to use mythffmpeg instead of ffmpeg. I didn’t see any differences when using it.

It does have a lot of commenting in it, so if you find problems, the log for it is our friend!

I use the following as a user job in mythtv:

/usr/local/bin/ffcut.py %DIR% %CHANID% %STARTTIMEUTC% NO >> /tmp/ffcut.log

the NO above is important as it keeps a copy of the original video as .ts.orig .


https://pastebin.com/UhW637Wv?

Hope this is useful.

Regards!

Jay

On 08/06/2023 20:10, Jay Harbeston wrote:
>
> Here is the first stab at the python version. It mimics the operation of
> the ffcut.sh, without all the db issues earlier mentioned, and uses the
> MythDB connection simplifying connection to the DB. I did change it to
> use mythffmpeg instead of ffmpeg. I didn’t see any differences when
> using it.
>
> It does have a lot of commenting in it, so if you find problems, the log
> for it is our friend!
>
> I use the following as a user job in mythtv:
>
> /usr/local/bin/ffcut.py %DIR% %CHANID% %STARTTIMEUTC% NO >> /tmp/ffcut.log
>
> the NO above is important as it keeps a copy of the original video as
> .ts.orig .
>
>
> ffcut.py <https://pastebin.com/UhW637Wv>
> pastebin.com <https://pastebin.com/UhW637Wv>
> favicon.ico <https://pastebin.com/UhW637Wv>
>
> <https://pastebin.com/UhW637Wv>
>
> Hope this is useful.
>
> Regards!
>
> Jay

It's certainly interesting to see how this job might be done in python,
but I'm a little concerned about its direct modification of the DB.

There are four mysqlupd instances and although they may well work as
intended I think I would be careful about using it on a system with a
'valuable' database.

Just saying :-)

John P

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

-----Original message-----
From: Jay Harbeston <jharbestonus@gmail.com>
Sent: Thursday 8th June 2023 15:58
To: Discussion about MythTV <mythtv-users@mythtv.org>
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings



Can you do the following command on the file that has the audio go away? : 

ffprobe -i  file.ts


And post the results in a reply. I want to see how the audio streams get defined by ffmpeg.

Jay

Sadly I don't have any of those files available now, but when I was checking them the other day, there was no audio track reported by mediainfo or vlc player. When I was watching the encoding, I could see that audio was reported as zero bytes

There may be a copy of the original file saved as  file.ts.orig in your recordings directory  if you didn’t add the the dontremove flag on the ffcut command . 

Jay



I'm just transcoding another file with your ffmpeg commands - this is the ff0.log contents:

dave@mythtv:/tmp/myth297314$ tail -f ff0.log 

  Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0 channels, fltp

Output #0, mpegts, to '/tmp/myth297314/tmpfile0.ts':

  Metadata:

    encoder         : Lavf59.27.100

  Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 24 tbr, 90k tbn

Stream mapping:

  Stream #0:0 -> #0:0 (copy)

Press [q] to stop, [?] for help

frame=130215 fps=5553 q=-1.0 Lsize= 5446811kB time=01:30:25.95 bitrate=8223.5kbits/s speed= 231x    

video:5298670kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.795805%



ffprobe on the original file for the above transcode looks like this:


dave@mythtv:/tmp/myth297314$ ffprobe /media/myth/tv/81003_20230529172000.ts

ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers

  built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)

 (Configuration details removed to reduce size of email message)



[mpegts @ 0x55a29ff90bc0] start time for stream 1 is not set in estimate_timings_from_pts

[mpegts @ 0x55a29ff90bc0] stream 1 : no TS found at start of file, duration not set

[mpegts @ 0x55a29ff90bc0] Could not find codec parameters for stream 1 (Audio: ac3 ([129][0][0][0] / 0x0081), 0 channels, fltp): unspecified sample rate

Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

Input #0, mpegts, from '/media/myth/tv/81003_20230529172000.ts':

  Duration: 01:42:11.58, start: 3.014378, bitrate: 8717 kb/s

  Program 1 

  Stream #0:0[0x1011]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps, 24 tbr, 90k tbn

  Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0 channels, fltp



And this is the ffprobe for the transcoded file:


dave@mythtv:/tmp/myth297314$ ffprobe tmpfile0.ts 

ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers

  built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)






Input #0, mpegts, from 'tmpfile0.ts':

  Duration: 01:30:25.67, start: 1.817000, bitrate: 8223 kb/s

  Program 1 

    Metadata:

      service_name    : Service01

      service_provider: FFmpeg

  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps, 24 tbr, 90k tbn







Regards,

Dave.

On 09/06/2023 08:00, Lists wrote:
> -----Original message-----
> *From:* Jay Harbeston <jharbestonus@gmail.com>
> *Sent:* Thursday 8th June 2023 15:58
> *To:* Discussion about MythTV <mythtv-users@mythtv.org>
> *Subject:* Re: [mythtv-users] Transcode script that keeps show in
> Recordings
>
>
>
> Can you do the following command on the file that has the
> audio go away? :
>
> ffprobe -i  file.ts
>
>
> And post the results in a reply. I want to see how the audio
> streams get defined by ffmpeg.
>
> Jay
>
>
> Sadly I don't have any of those files available now, but when I
> was checking them the other day, there was no audio track
> reported by mediainfo or vlc player. When I was watching the
> encoding, I could see that audio was reported as zero bytes
>
>
> There may be a copy of the original file saved as  file.ts.orig in
> your recordings directory  if you didn’t add the the dontremove flag
> on the ffcut command .
>
> Jay
>
>
> I'm just transcoding another file with your ffmpeg commands - this is
> the ff0.log contents:
>
> *dave@mythtv*:*/tmp/myth297314*$ tail -f ff0.log
>
>   Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0
> channels, fltp
>
> Output #0, mpegts, to '/tmp/myth297314/tmpfile0.ts':
>
>   Metadata:
>
>     encoder         : Lavf59.27.100
>
>   Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv,
> bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 24
> tbr, 90k tbn
>
> Stream mapping:
>
>   Stream #0:0 -> #0:0 (copy)
>
> Press [q] to stop, [?] for help
>
> frame=130215 fps=5553 q=-1.0 Lsize= 5446811kB time=01:30:25.95
> bitrate=8223.5kbits/s speed= 231x
>
> video:5298670kB audio:0kB subtitle:0kB other streams:0kB global
> headers:0kB muxing overhead: 2.795805%
>
>
> ffprobe on the original file for the above transcode looks like this:
>
> *dave@mythtv*:*/tmp/myth297314*$ ffprobe
> /media/myth/tv/81003_20230529172000.ts
>
> ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers
>
>   built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)
>
>  (Configuration details removed to reduce size of email message)
>
>
> [mpegts @ 0x55a29ff90bc0] start time for stream 1 is not set in
> estimate_timings_from_pts
>
> [mpegts @ 0x55a29ff90bc0] stream 1 : no TS found at start of file,
> duration not set
>
> [mpegts @ 0x55a29ff90bc0] Could not find codec parameters for stream 1
> (Audio: ac3 ([129][0][0][0] / 0x0081), 0 channels, fltp): unspecified
> sample rate
>
> Consider increasing the value for the 'analyzeduration' (0) and
> 'probesize' (5000000) options
>
> Input #0, mpegts, from '/media/myth/tv/81003_20230529172000.ts':
>
>   Duration: 01:42:11.58, start: 3.014378, bitrate: 8717 kb/s
>
>   Program 1
>
>   Stream #0:0[0x1011]: Video: h264 (High) ([27][0][0][0] / 0x001B),
> yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps,
> 24 tbr, 90k tbn
>
>   Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0
> channels, fltp
>
>
> And this is the ffprobe for the transcoded file:
>
> *dave@mythtv*:*/tmp/myth297314*$ ffprobe tmpfile0.ts
>
> ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers
>
>   built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)
>
>
>
> Input #0, mpegts, from 'tmpfile0.ts':
>
>   Duration: 01:30:25.67, start: 1.817000, bitrate: 8223 kb/s
>
>   Program 1
>
>     Metadata:
>
>       service_name    : Service01
>
>       service_provider: FFmpeg
>
>   Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B),
> yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps,
> 24 tbr, 90k tbn
>
>
>
>
> Regards,
>
> Dave.

I get that 'deja vu all over again' feeling. Perhaps try

mythffmpeg -fflags +genpts -i inputfile -acodec copy -vcodec copy
-scodec copy -f mpegts outputfile

and rebuild the seektable. Then run the script again.

https://github.com/bennettpeter/android-MythTV-Leanfront/issues/54#issuecomment-1368201782

https://github.com/MythTV/mythtv/issues/600

HTH

John P
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

-----Original message-----
From: John Pilkington <johnpilk222@gmail.com>
Sent: Friday 9th June 2023 9:34
To: mythtv-users@mythtv.org
Subject: Re: [mythtv-users] Transcode script that keeps show in Recordings


On 09/06/2023 08:00, Lists wrote:
> -----Original message-----
> *From:* Jay Harbeston <jharbestonus@gmail.com>
> *Sent:* Thursday 8th June 2023 15:58
> *To:* Discussion about MythTV <mythtv-users@mythtv.org>
> *Subject:* Re: [mythtv-users] Transcode script that keeps show in
> Recordings
>
>
>
> Can you do the following command on the file that has the
> audio go away? :
>
> ffprobe -i file.ts
>
>
> And post the results in a reply. I want to see how the audio
> streams get defined by ffmpeg.
>
> Jay
>
>
> Sadly I don't have any of those files available now, but when I
> was checking them the other day, there was no audio track
> reported by mediainfo or vlc player. When I was watching the
> encoding, I could see that audio was reported as zero bytes
>
>
> There may be a copy of the original file saved as file.ts.orig in
> your recordings directory if you didn't add the the dontremove flag
> on the ffcut command .
>
> Jay
>
>
> I'm just transcoding another file with your ffmpeg commands - this is
> the ff0.log contents:
>
> *dave@mythtv*:*/tmp/myth297314*$ tail -f ff0.log
>
> Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0
> channels, fltp
>
> Output #0, mpegts, to '/tmp/myth297314/tmpfile0.ts':
>
> Metadata:
>
> encoder : Lavf59.27.100
>
> Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv,
> bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 24
> tbr, 90k tbn
>
> Stream mapping:
>
> Stream #0:0 -> #0:0 (copy)
>
> Press [q] to stop, [?] for help
>
> frame=130215 fps=5553 q=-1.0 Lsize= 5446811kB time=01:30:25.95
> bitrate=8223.5kbits/s speed= 231x
>
> video:5298670kB audio:0kB subtitle:0kB other streams:0kB global
> headers:0kB muxing overhead: 2.795805%
>
>
> ffprobe on the original file for the above transcode looks like this:
>
> *dave@mythtv*:*/tmp/myth297314*$ ffprobe
> /media/myth/tv/81003_20230529172000.ts
>
> ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers
>
> built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)
>
> (Configuration details removed to reduce size of email message)
>
>
> [mpegts @ 0x55a29ff90bc0] start time for stream 1 is not set in
> estimate_timings_from_pts
>
> [mpegts @ 0x55a29ff90bc0] stream 1 : no TS found at start of file,
> duration not set
>
> [mpegts @ 0x55a29ff90bc0] Could not find codec parameters for stream 1
> (Audio: ac3 ([129][0][0][0] / 0x0081), 0 channels, fltp): unspecified
> sample rate
>
> Consider increasing the value for the 'analyzeduration' (0) and
> 'probesize' (5000000) options
>
> Input #0, mpegts, from '/media/myth/tv/81003_20230529172000.ts':
>
> Duration: 01:42:11.58, start: 3.014378, bitrate: 8717 kb/s
>
> Program 1
>
> Stream #0:0[0x1011]: Video: h264 (High) ([27][0][0][0] / 0x001B),
> yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps,
> 24 tbr, 90k tbn
>
> Stream #0:1[0x1100]: Audio: ac3 ([129][0][0][0] / 0x0081), 0
> channels, fltp
>
>
> And this is the ffprobe for the transcoded file:
>
> *dave@mythtv*:*/tmp/myth297314*$ ffprobe tmpfile0.ts
>
> ffprobe version 5.1.2-3ubuntu1 Copyright (c) 2007-2022 the FFmpeg developers
>
> built with gcc 12 (Ubuntu 12.2.0-14ubuntu2)
>
>
>
> Input #0, mpegts, from 'tmpfile0.ts':
>
> Duration: 01:30:25.67, start: 1.817000, bitrate: 8223 kb/s
>
> Program 1
>
> Metadata:
>
> service_name : Service01
>
> service_provider: FFmpeg
>
> Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B),
> yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 24 fps,
> 24 tbr, 90k tbn
>
>
>
>
> Regards,
>
> Dave.

I get that 'deja vu all over again' feeling. Perhaps try

mythffmpeg -fflags +genpts -i inputfile -acodec copy -vcodec copy
-scodec copy -f mpegts outputfile

and rebuild the seektable. Then run the script again.

https://github.com/bennettpeter/android-MythTV-Leanfront/issues/54#issuecomment-1368201782

https://github.com/MythTV/mythtv/issues/600

HTH

John P
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Thanks John,



Your mythffmpeg command line seems to do the trick - I can now cut the adverts form my recordings and rebuild the seektable after the transcode.



I've been modifying Jay Harbeston's ffcut.sh script, since when I first tried it, it didn't work at all with my h264 recordings - I also had to modify his sql commands, since they didn't work for me (as the user I ran the script as, in testing, does not have a mysql account), then I had to modify the ffmpeg commands for h264. My final task, since someone suggested that it isn't a good idea to directly modify the sql database from these scripts will be to use mythutil and mythcommflag in place of the sql commands.



I'll be happy to share my version of the script, so long as Jay has no objections (I'll make sure I'll credit him in the script for the 99% of the work he did)



Regards,

Dave.

> On 9 Jun 2023, at 8:56 pm, Lists <lists@the-pearsons.co.uk> wrote:
>
> Thanks John,
>
> Your mythffmpeg command line seems to do the trick - I can now cut the adverts form my recordings and rebuild the seektable after the transcode.
>
> I've been modifying Jay Harbeston's ffcut.sh script, since when I first tried it, it didn't work at all with my h264 recordings - I also had to modify his sql commands, since they didn't work for me (as the user I ran the script as, in testing, does not have a mysql account), then I had to modify the ffmpeg commands for h264. My final task, since someone suggested that it isn't a good idea to directly modify the sql database from these scripts will be to use mythutil and mythcommflag in place of the sql commands.
>
> I'll be happy to share my version of the script, so long as Jay has no objections (I'll make sure I'll credit him in the script for the 99% of the work he did)
>
> Regards,
> Dave.

Dave I've got and you're welcome to if you like, an h264cut.sh. Someone here tried it and had problems, but for me it has worked flawlessly for years.
I transcode to a .ts container as I host the odd snippet (*) and .ts lets shotcut run without lip-sync problems. I use shotcut to move (no pun) the mov atom to the head to allow you to watch in a browser.
James

* https://tigger.ws/downloads/qna.mp4