Mailing List Archive

Hi Peter

Having looked at framesPlayed a little more - I think we are trying to
do too much with one variable - which is bound to fail at some point.
I plan on using a distinct variable in VideoFrame to track
discontinuities which should make things a little cleaner.

With respect to fpsMultiplier, I suspect MediaCodec is deinterlacing
adaptively - at least for some content. So in certain circumstances
(e.g. telecined material) it probably returns a mixture of raw
progressive and deinterlaced progressive frames which may result in a
variable frame rate? I've noticed that the fpsMultiplier code gets
triggered repeatedly for VAAPI decoder deinterlacing (in the render
branch) with certain material - again I think telecined.

That said, I can't think of an alternative to detecting what
MediaCodec is returning - unless there is something in the portions of
the API that FFmpeg exposes?

Regards
Mark

On Sun, 30 Jun 2019 at 19:21, Peter Bennett <pb.mythtv@gmail.com> wrote:
>
> Hi Mark
>
> I am looking into different ways to fix the bug
> https://code.mythtv.org/trac/ticket/13416
>
> There is another area that may cause you problems with framesPlayed.
>
> The shield has frame-doubling deinterlace built in, and the result was
> the elapsed time was being calculated at double the correct rate. This
> had bad consequences for many things. I added some code in
> avformatdecoder (line 4123 onwards), to detect that frame-doubling was
> being done. It sets m_fpsMultiplier to 2 and then uses logic in line
> 4160 to only increment framesPlayed on every alternate frame. Per the
> ticket, this code is not working correctly as it seems that now the
> timing is sometimes off by a factor of 1.5 instead of a factor of 2.
> This may also affect your usage of framesPlayed in the render. You may
> need to revert this code as well.
>
> Let me know if you revert that. I will probably revert it anyway when I
> have figured out a real fix for the ticket 13416.
>
> Regards
> Peter
>
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Hi Mark

I assume that means we can reinstate the lines you commented? I could
make them a bit more intelligent, by only adjust framesPlayed when it
goes off by more than a certain amount, which would reduce discontinuities.

I think I should remove the fpsMultiplier logic - it is getting wrong
results and then they are overwritten by the AVSync2 logic.

Reagrds
Peter


On 7/1/19 3:54 PM, Mark Kendall wrote:
> Hi Peter
>
> Having looked at framesPlayed a little more - I think we are trying to
> do too much with one variable - which is bound to fail at some point.
> I plan on using a distinct variable in VideoFrame to track
> discontinuities which should make things a little cleaner.
>
> With respect to fpsMultiplier, I suspect MediaCodec is deinterlacing
> adaptively - at least for some content. So in certain circumstances
> (e.g. telecined material) it probably returns a mixture of raw
> progressive and deinterlaced progressive frames which may result in a
> variable frame rate? I've noticed that the fpsMultiplier code gets
> triggered repeatedly for VAAPI decoder deinterlacing (in the render
> branch) with certain material - again I think telecined.
>
> That said, I can't think of an alternative to detecting what
> MediaCodec is returning - unless there is something in the portions of
> the API that FFmpeg exposes?
>
> Regards
> Mark
>
> On Sun, 30 Jun 2019 at 19:21, Peter Bennett <pb.mythtv@gmail.com> wrote:
>> Hi Mark
>>
>> I am looking into different ways to fix the bug
>> https://code.mythtv.org/trac/ticket/13416
>>
>> There is another area that may cause you problems with framesPlayed.
>>
>> The shield has frame-doubling deinterlace built in, and the result was
>> the elapsed time was being calculated at double the correct rate. This
>> had bad consequences for many things. I added some code in
>> avformatdecoder (line 4123 onwards), to detect that frame-doubling was
>> being done. It sets m_fpsMultiplier to 2 and then uses logic in line
>> 4160 to only increment framesPlayed on every alternate frame. Per the
>> ticket, this code is not working correctly as it seems that now the
>> timing is sometimes off by a factor of 1.5 instead of a factor of 2.
>> This may also affect your usage of framesPlayed in the render. You may
>> need to revert this code as well.
>>
>> Let me know if you revert that. I will probably revert it anyway when I
>> have figured out a real fix for the ticket 13416.
>>
>> Regards
>> Peter
>>
>>
>> _______________________________________________
>> mythtv-dev mailing list
>> mythtv-dev@mythtv.org
>> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
>> http://wiki.mythtv.org/Mailing_List_etiquette
>> MythTV Forums: https://forum.mythtv.org
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Mon, Jul 01, 2019 at 04:06:28PM -0400, Peter Bennett wrote:
> Hi Mark
>
> I assume that means we can reinstate the lines you commented? I could make
> them a bit more intelligent, by only adjust framesPlayed when it goes off by
> more than a certain amount, which would reduce discontinuities.
>
> I think I should remove the fpsMultiplier logic - it is getting wrong
> results and then they are overwritten by the AVSync2 logic.

Does ffmpeg allow us to attach private data to each frame to be
decoded and have it returned back to us after decoding? If so, we
could put our framesPlayed value there. If the frame gets 2x
deintelaced, we'd get back two frames with the same framesPlayed
value. Would that be a problem? Alternatively, we could put our own
copy of the frame timestamp there. If it gets 2x deinterlaced, the
second frame would come back with a slightly later timestamp than what
our returned value, right? If so, could not count those frames toward
framesPlayed.

David

> Reagrds
> Peter
>
>
> On 7/1/19 3:54 PM, Mark Kendall wrote:
> > Hi Peter
> >
> > Having looked at framesPlayed a little more - I think we are trying to
> > do too much with one variable - which is bound to fail at some point.
> > I plan on using a distinct variable in VideoFrame to track
> > discontinuities which should make things a little cleaner.
> >
> > With respect to fpsMultiplier, I suspect MediaCodec is deinterlacing
> > adaptively - at least for some content. So in certain circumstances
> > (e.g. telecined material) it probably returns a mixture of raw
> > progressive and deinterlaced progressive frames which may result in a
> > variable frame rate? I've noticed that the fpsMultiplier code gets
> > triggered repeatedly for VAAPI decoder deinterlacing (in the render
> > branch) with certain material - again I think telecined.
> >
> > That said, I can't think of an alternative to detecting what
> > MediaCodec is returning - unless there is something in the portions of
> > the API that FFmpeg exposes?
> >
> > Regards
> > Mark
> >
> > On Sun, 30 Jun 2019 at 19:21, Peter Bennett <pb.mythtv@gmail.com> wrote:
> > > Hi Mark
> > >
> > > I am looking into different ways to fix the bug
> > > https://code.mythtv.org/trac/ticket/13416
> > >
> > > There is another area that may cause you problems with framesPlayed.
> > >
> > > The shield has frame-doubling deinterlace built in, and the result was
> > > the elapsed time was being calculated at double the correct rate. This
> > > had bad consequences for many things. I added some code in
> > > avformatdecoder (line 4123 onwards), to detect that frame-doubling was
> > > being done. It sets m_fpsMultiplier to 2 and then uses logic in line
> > > 4160 to only increment framesPlayed on every alternate frame. Per the
> > > ticket, this code is not working correctly as it seems that now the
> > > timing is sometimes off by a factor of 1.5 instead of a factor of 2.
> > > This may also affect your usage of framesPlayed in the render. You may
> > > need to revert this code as well.
> > >
> > > Let me know if you revert that. I will probably revert it anyway when I
> > > have figured out a real fix for the ticket 13416.
> > >
> > > Regards
> > > Peter
> > >
> > >
> > > _______________________________________________
> > > mythtv-dev mailing list
> > > mythtv-dev@mythtv.org
> > > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > > http://wiki.mythtv.org/Mailing_List_etiquette
> > > MythTV Forums: https://forum.mythtv.org
> > _______________________________________________
> > mythtv-dev mailing list
> > mythtv-dev@mythtv.org
> > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > http://wiki.mythtv.org/Mailing_List_etiquette
> > MythTV Forums: https://forum.mythtv.org
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Mon, 1 Jul 2019 at 21:07, Peter Bennett <pb.mythtv@gmail.com> wrote:
>
> Hi Mark
>
> I assume that means we can reinstate the lines you commented? I could
> make them a bit more intelligent, by only adjust framesPlayed when it
> goes off by more than a certain amount, which would reduce discontinuities.

Yes- I'm going to reinstate those lines.
Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Mon, 1 Jul 2019 at 22:13, David Engel <david@istwok.net> wrote:
> Does ffmpeg allow us to attach private data to each frame to be
> decoded and have it returned back to us after decoding? If so, we
> could put our framesPlayed value there. If the frame gets 2x
> deintelaced, we'd get back two frames with the same framesPlayed
> value. Would that be a problem? Alternatively, we could put our own
> copy of the frame timestamp there. If it gets 2x deinterlaced, the
> second frame would come back with a slightly later timestamp than what
> our returned value, right? If so, could not count those frames toward
> framesPlayed.

Hi David

That's a useful idea and would probably work if we needed something
for other decoders.

As far as I can tell, however, we never really have visibility of the
mediacodec frames before they come out of the decoder. Mediacodec
handles all of its buffering internally and for both direct rendering
and copy back decoding, the first we see of the frame is in
AvFormatDecoder::ProcessVideoFrame - which is the same point that we
process framesPlayed. I may be missing something but I can't currently
see it working.

The other solution would be to add some private data to the packets
fed into the decoder but AVPacket has no opaque field for such a use
case and it probably wouldn't be reliable - as frames are often
returned out of order.

Regards
Mark
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org