Mailing List Archive

Tim

A lot has changed internally with video buffers, frame sizing, alignment etc.

Essentially in latest master, we try and use the actual size of the
video frame, use some alignment for allocating memory to give
ourselves a little wiggle room with FFmpeg's 'quirks' and otherwise
ignore values such as 1088 (other than telling the display classes to
ignore the extra rows).

Previously the video frame size was rounded up to ensure the
dimensions were divisible by 16 - and in this way ensure memory
allocations were enough for FFmpeg (but causing sundry other
complexities).

So in your case, 1080p video is now properly marked as having a height of 1080.

I can think of 2 possible issues that you might be seeing:-

FFmpeg likes to do its own alignment internally - which becomes
problematic when we start to use libavfilter etc (and in particular
VAAPI filters). So previously FFmpeg would have been told the incoming
video was 1088 and been happy. Now it rounds up to 1088 and, at a
guess, decides that it needs to do some scaling and/or conversion
because 1080 != 1088. There is a similar problem with the VAAPI VPP
deinterlacing in libmythtv (that I haven't figured out how to solve
yet).

I suspect disabling the VAAPI deinterlacer may produce useful output
(though it may be going wrong elsewhere as well).

The other possibility is that memory allocations/alignment are not
what FFmpeg is expecting and hence the fifo transfers are corrupted.
(see line 887 onwards in mythtranscode/transcode.cpp). I maintained
existing behaviour when refactoring the allocations - though on
reflection it is possible that the call to GetAlignedBuffer at line
911 is throwing things out (as it adds a hardcoded extra 64bytes of
memory for each allocated frame).

Hopefully this helps - happy to make changes/fixes if needed - but to
be honest, I've had my eye on removing the fifo functionality (as I
assumed it was only used by the largely deceased nuvexport) and was
looking at just adding full FFmpeg functionality to transcode
directly.

regards
Mark



On Thu, 26 Dec 2019 at 16:39, Tim Pletcher <pletchtd@gmail.com> wrote:
>
> I have a script that I use to crop recording cutlist and transcode to h264 using VAAPI acceleration via ffmpeg. Now that I have been running master, I noticed the behavior of mythtranscode --fifoinfo has changed.
>
> On v30-fixes, a recorded video from a 1920x1080 channel produces the following excerpted output:
> command: mythtranscode --chanid ' + chanid + ' --starttime ' + utc_start_time + ' --fifoinfo
>
> 2019-12-26 10:21:52.170921 N Found video height of 1088. This is unusual and more than likely the video is actually 1080 so mythtranscode will treat it as such.
>
> 2019-12-26 10:21:52.224561 I FifoVideoWidth 1920
>
> 2019-12-26 10:21:52.224567 I FifoVideoHeight 1088
>
> 2019-12-26 10:21:52.224573 I FifoVideoAspectRatio 1.77778
>
> 2019-12-26 10:21:52.224575 I FifoVideoFrameRate 29.97
>
> 2019-12-26 10:21:52.224577 I FifoAudioFormat raw
>
> 2019-12-26 10:21:52.224577 I FifoAudioChannels 2
>
> 2019-12-26 10:21:52.224579 I FifoAudioSampleRate 48000
>
>
> On master, the same command and recording produces:
>
> 2019-12-26 10:34:43.176749 I FifoVideoWidth 1920
>
> 2019-12-26 10:34:43.176755 I FifoVideoHeight 1080
>
> 2019-12-26 10:34:43.176761 I FifoVideoAspectRatio 1.77778
>
> 2019-12-26 10:34:43.176764 I FifoVideoFrameRate 29.97
>
> 2019-12-26 10:34:43.176765 I FifoAudioFormat raw
>
> 2019-12-26 10:34:43.176767 I FifoAudioChannels 2
>
> 2019-12-26 10:34:43.176768 I FifoAudioSampleRate 48000
>
>
> Once I retrieve this information, I use mythtranscode to then pipe the video/audio to ffmpeg via a command like this:
> mythtranscode --passthrough --cleancut --honorcutlist --chanid ' + chanid + ' --starttime ' + utc_start_time + ' --fifodir ' + args.output_dir
>
> The new behavior creates an issue because I parse the earlier reported Fifo information to structure the ffmpeg command in my script:
> Example:
> ffmpeg -y -init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device intel -filter_hw_device intel -thread_queue_size 3072 -i audout -f rawvideo -s:v 1920x1088 -r 29.97 -thread_queue_size 3072 -i vidout -vf 'format=nv12|vaapi,hwupload,deinterlace_vaapi=mode=motion_compensated,deinterlace_vaapi=rate=field' -c:v h264_vaapi -b:v 4M -maxrate 6M -c:a libfdk_aac -filter:a 'volume=1.5' -ac 2 -b:a 320k output.mp4
>
> The problem I am experiencing is that the -s:v aspect ratio parameter must be specified with rawvideo and a value of 1920x1080 as reported from master produces corrupted transcoded video output from ffmpeg (example: video) while using the 1920x1088 as reported from v30-fixes works fine. If I take the original .ts recording and use ffmpeg to analyze the media information, it reports that the video as recorded by the backend is truly 1920x1080.
>
> Is the fifo stream produced from mythtranscode actually 1920x1088 despite reporting 1920x1080 possibly due to some sort of mod16 rather than mod8 encoding?
>
> -Tim
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 26/12/2019 16:38, Tim Pletcher wrote:
> I have a script that I use to crop recording cutlist and transcode to
> h264 using VAAPI acceleration via ffmpeg.  Now that I have been running
> master, I noticed the behavior of mythtranscode --fifoinfo has changed.

Does your reference just to frame-size reporting mean that in every
other way transcoding this way still works? In particular, is cleancut
still working? Seeing all the changes being made to the rendering
mechanism over recent times, I'd imagined cleancut was sure to become
broken. I'll be very pleased if it still works.

(For various reasons, I'm stuck on v29 so haven't tested any of this
myself).

Cheers,
Paul.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

>
>
>
> Does your reference just to frame-size reporting mean that in every
> other way transcoding this way still works? In particular, is cleancut
> still working? Seeing all the changes being made to the rendering
> mechanism over recent times, I'd imagined cleancut was sure to become
> broken. I'll be very pleased if it still works.
>
>
> Yes, if i override / specify a height of 1088 to ffmpeg for the incoming
fifo rawvideo stream, everything including cleancut seems to work fine.
For now, I will just do this in my transcoding script.

Tim

> On 3 Jan 2020, at 10:26 pm, mythtv-dev-request@mythtv.org wrote:
>
>> I have a script that I use to crop recording cutlist and transcode to
>> h264 using VAAPI acceleration via ffmpeg.? Now that I have been running
>> master, I noticed the behavior of mythtranscode --fifoinfo has changed.
>
> Does your reference just to frame-size reporting mean that in every
> other way transcoding this way still works? In particular, is cleancut
> still working? Seeing all the changes being made to the rendering
> mechanism over recent times, I'd imagined cleancut was sure to become
> broken. I'll be very pleased if it still works.
>
> (For various reasons, I'm stuck on v29 so haven't tested any of this
> myself).


Paul likewise I have a script that has not worked of late.
I was about to go exploring, but if you’d not mind sending me your script it would save me hard work

Cheers
James

As good faith i put my script here. I’m sure I started with someones efforts, but any links are long gone, so if any of this looks familiar appologies and thanks

[sandypit] /store/jam [989]% cat bin/h264cut.sh
#!/bin/bash
#Uses mythcommflag, ffmpeg, and mkvmerge to cut commercials out of h.264 encoded mpg files#
#set -x

echo -------------------------------------------------------
echo -n "recording eg 20181012 being /store/*201811012*.ts: "
read recname
echo -------------------------------------------------------
ls -lh /store/*$recname*.ts

echo -------------------------------------------------------
echo -n "Choose which file: "
read recname
echo -------------------------------------------------------
echo -n "now the name for the recording: "
read vidname
echo -------------------------------------------------------
if [[ $vidname == *\.* ]]; then
usename=$vidname
else
usename=$vidname".mp4";
fi

echo now choose the destinatin dir: $usename
echo 0=MythSeasons
echo 1=MythSeries
echo -n A D G J M P S V or \<cr\> for the prefix:
read prefix

case $prefix in
0)
dir="/store/Movies/MythSeasons";
;;
1)
dir="/store/Movies/MythSeries";
;;
[Aa])
dir="/store/Movies/MythA-C";
;;
[Dd])
dir="/store/Movies/MythD-F";
;;
[Gg])
dir="/store/Movies/MythG-I";
;;
[Jj])
dir="/store/Movies/MythJ-L";
;;
[Mm])
dir="/store/Movies/MythM-O";
;;
[Pp])
dir="/store/Movies/MythP-R";
;;
[Ss])
dir="/store/Movies/MythS-U";
;;
[Vv])
dir="/store/Movies/MythV-Z";
;;
*)
prefix=`echo $vidname | head -c 1`
case $prefix in
[0123456789])
dir="/store/Movies/Myth0-9"
;;
[ABCabc])
dir="/store/Movies/MythA-C"
;;
[DEFdef])
dir="/store/Movies/MythD-F"
;;
[GHIghi])
dir="/store/Movies/MythG-I"
;;
[JKLjkl])
dir="/store/Movies/MythJ-L"
;;
[MNOmno])
dir="/store/Movies/MythM-O"
;;
[PQRpqr])
dir="/store/Movies/MythP-R"
;;
[STUstu])
dir="/store/Movies/MythS-U"
;;
[VWXYZvwyzx])
dir="/store/Movies/MythV-Z"
;;
esac
;;
esac
echo -------------------------------------------------------

#cleanup $workdir
workdir="/store/TScut_work"
rm ${workdir}/*
echo dst:$dir/$usename
echo -------------------------------------------------------

if [ -z "`ls ${workdir}`" ]; then
mkdir -p ${workdir}
fi
if [ ! -f $recname ]; then
echo "[fatal] file doesn't exist, aborting."
exit 1
fi

DBHostName=localhost
DBUserName=mythtv
DBPassword=mythtv
DBName=mythconverg

#connect to DB
mysqlconnect="mysql -N -h$DBHostName -u$DBUserName -p$DBPassword $DBName"
export mysqlconnect

#deteremine directory and filename
RECDIR=`dirname $recname`
BASENAME=`basename $recname`
tmp=`basename $BASENAME .ts`

logfile=$workdir/${BASENAME}.log
>${logfile}

#determine chanid and starttime from recorded table
echo "select chanid from recorded where basename=\"$BASENAME\";"
chanid=`echo "select chanid from recorded where basename=\"$BASENAME\";" | $mysqlconnect`

echo "select starttime from recorded where basename=\"$BASENAME\";"
starttime=`echo "select starttime from recorded where basename=\"$BASENAME\";" | $mysqlconnect`

#determine FPS (frames per second) from db, which is used to determine the seek time to extract video clips
fps=$(echo "scale=10;`echo "select data from recordedmarkup where chanid=$chanid AND starttime='$starttime' and type=32" | $mysqlconnect` / 1000.0" | bc)

if [ -z "$chanid" ] || [ -z "$starttime" ]
then
echo "[fatal] Recording not found in MythTV database, script aborted."
exit 1
elif [ -z "$fps" ]
then
echo "[fatal] Frames per second value not found in MythTV database, script aborted."
exit 1
fi

#lets make sure we have a cutlist before we proceed
if [. -z "`mythutil -q -q --getcutlist --chanid $chanid --starttime "$starttime" | sed 's/Cutlist: $//'`" ]; then
echo "[fatal] no cutlist found"
exit 1
fi

#cutlist provides a list of frames in the format start-end,[start1-end1,....] to cut
#we swap this list so that it provides the ranges of video we want in the format
# was cut 100-200 600-700 list is now 100,200-600,700-end singe play ie 100 implies 0-100
CUTLIST=`mythutil -q -q --getcutlist --chanid $chanid --starttime "$starttime" | sed 's/Cutlist: //' | \
sed 's/-/ /g' | sed 's/^\|$/-/g' | sed 's/,/-/g'`

clipcount=0
for i in ${CUTLIST}
do
start=`echo $i | sed 's/ //g' | sed 's/^\(.*\)-.*$/\1/'`
end=`echo $i | sed 's/ //g' | sed 's/^.*-\(.*\)$/\1/'`

#if $start is empty, deal with it
if [ -z $start ]; then
#set $start to 0
start=0
if [ $start -eq $end ]; then
continue
fi
fi
#convert start into time in seconds (divide frames by frames per second)
start=$(echo "scale=8; $start / $fps" | bc -l)
#if $end is not null, we can do things

if [ -n "$end" ]; then
clipcount=$((++clipcount))
end=$(echo "scale=8; $end / $fps" | bc -l)
duration=`echo "$end - $start" | bc -l`

echo "[ffmpeg] -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss $start -t $duration ${workdir}/${tmp}_${clipcount}.mp4" &>>${logfile}
echo "[ffmpeg] -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss $start -t $duration ${workdir}/${tmp}_${clipcount}.mp4"
ffmpeg -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss $start -t $duration ${workdir}/${tmp}_${clipcount}.mp4 &>>${logfile}
#echo "file ${workdir}/${tmp}_${clipcount}.mp4"
echo "file ${workdir}/${tmp}_${clipcount}.mp4" >> ${workdir}/file

elif [ -z "$end" ]; then
clipcount=$((++clipcount))
echo "{ffmpeg} -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss ${start} ${workdir}/${tmp}_${clipcount}.mp4" &>>${logfile}
echo "{ffmpeg} -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss ${start} ${workdir}/${tmp}_${clipcount}.mp4"

ffmpeg -i ${RECDIR}/${BASENAME} -acodec copy -vcodec copy -ss ${start} ${workdir}/${tmp}_${clipcount}.mp4 &>>${logfile}
echo #"file ${workdir}/${tmp}_${clipcount}.mp4"
echo "file ${workdir}/${tmp}_${clipcount}.mp4" >> ${workdir}/file
fi
done

echo
echo "[ffmpeg] -f concat -safe 0 -i $workdir/file -c copy $dir/$usename" &>>${logfile}
echo "[ffmpeg] -f concat -safe 0 -i $workdir/file -c copy $dir/$usename"
ffmpeg -f concat -safe 0 -i $workdir/file -c copy "$dir/$usename" &>> $logfile

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org