Mailing List Archive

Call for testing branch devel/ffmpeg-resync (mpegts-harmonize)
Hello all,

Following the merge of my FFmpeg cleanup commits to master, branch
devel/ffmpeg-resync now contains my changes harmonizing our version of
the mpegts demuxer with FFmpeg 4.4's version.

The main change versus FFmpeg that I couldn't test is DSMCC/MHEG, so
could someone with access to those streams confirm that it works the
same as master?

Other testing is, of course, also welcome.

Thanks,

Scott Theisen
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 23/07/2022 20:21, Scott Theisen wrote:
> Hello all,
>
> Following the merge of my FFmpeg cleanup commits to master, branch
> devel/ffmpeg-resync now contains my changes harmonizing our version of
> the mpegts demuxer with FFmpeg 4.4's version.
>
> The main change versus FFmpeg that I couldn't test is DSMCC/MHEG, so
> could someone with access to those streams confirm that it works the
> same as master?
>
> Other testing is, of course, also welcome.
>
> Thanks,
>
> Scott Theisen

Hi Scott: I have this running in F35 using the OpenGL profile, and have
repeated frontend crashes on quitting playback, either by <Esc> or at a
'cut to end' cutpoint. Logging via logpath doesn't look much help and I
haven't yet tried gdb.

On restarting the frontend after a crash the preview for that recording
is missing and stays missing, and there are Preview: <recordedid> lines
in the terminal log. Restarting the backend doesn't help. The
'unaligned fastbin chunk' reference happens fairly consistently.

John

{{{
2022-07-24 12:11:00.250601 I ScreenSaverX11: Inhibited X11 screensaver
2022-07-24 12:11:00.250704 I TV::HandleStateChange(): Main UI disabled.
2022-07-24 12:11:00.250759 I TV::StartTV(): Entering main playback loop.
2022-07-24 12:11:00.268545 I VideoOutput: SetDeinterlacing (Doublerate
1): Single Medium|CPU Double None
2022-07-24 12:11:00.268593 I MythDeint: Deinterlacer change: 0x0 None
dr:0 tff:1 -> 704x576 YUV420P dr:0 tff:1
2022-07-24 12:11:00.268876 I GLVid: New frame format: None:None 704x576
(Tex: 2D) -> YUV420P:YUV420P 704x576 (Tex: 2D)
2022-07-24 12:11:04.816738 E Preview: 13608:
2022-07-24 12:11:05.490364 E Preview: 13610:
malloc_consolidate(): unaligned fastbin chunk detected
Handling Aborted
Aborted (core dumped)
}}}
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 12:35, John Pilkington wrote:
> On 23/07/2022 20:21, Scott Theisen wrote:
>> Hello all,
>>
>> Following the merge of my FFmpeg cleanup commits to master, branch
>> devel/ffmpeg-resync now contains my changes harmonizing our version of
>> the mpegts demuxer with FFmpeg 4.4's version.
>>
>> The main change versus FFmpeg that I couldn't test is DSMCC/MHEG, so
>> could someone with access to those streams confirm that it works the
>> same as master?
>>
>> Other testing is, of course, also welcome.
>>
>> Thanks,
>>
>> Scott Theisen
>
> Hi Scott:  I have this running in F35 using the OpenGL profile, and have
> repeated frontend crashes on quitting playback, either by <Esc> or at a
> 'cut to end' cutpoint.  Logging via logpath doesn't look much help and I
> haven't yet tried gdb.
>
> On restarting the frontend after a crash the preview for that recording
> is missing and stays missing, and there are Preview: <recordedid>  lines
> in the terminal log.  Restarting the backend doesn't help.  The
> 'unaligned fastbin chunk' reference happens fairly consistently.
>
> John
>
> {{{
> 2022-07-24 12:11:00.250601 I  ScreenSaverX11: Inhibited X11 screensaver
> 2022-07-24 12:11:00.250704 I  TV::HandleStateChange(): Main UI disabled.
> 2022-07-24 12:11:00.250759 I  TV::StartTV(): Entering main playback loop.
> 2022-07-24 12:11:00.268545 I  VideoOutput: SetDeinterlacing (Doublerate
> 1): Single Medium|CPU Double None
> 2022-07-24 12:11:00.268593 I  MythDeint: Deinterlacer change: 0x0 None
> dr:0 tff:1 -> 704x576 YUV420P dr:0 tff:1
> 2022-07-24 12:11:00.268876 I  GLVid: New frame format: None:None 704x576
> (Tex: 2D) -> YUV420P:YUV420P 704x576 (Tex: 2D)
> 2022-07-24 12:11:04.816738 E  Preview: 13608:
> 2022-07-24 12:11:05.490364 E  Preview: 13610:
> malloc_consolidate(): unaligned fastbin chunk detected
> Handling Aborted
> Aborted (core dumped)
> }}}


I get the crash immediately on trying to run mythpreviewgen from the
console. I've seen this long ago, but it got fixed.
mythvideoprofile.cpp line 1325 reports No window!

The frontend is in a window in a kde plasma screen.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 16:21, John Pilkington wrote:

> On 24/07/2022 12:35, John Pilkington wrote:
>> On 23/07/2022 20:21, Scott Theisen wrote:
>>> Hello all,
>>>
>>> Following the merge of my FFmpeg cleanup commits to master, branch
>>> devel/ffmpeg-resync now contains my changes harmonizing our version
>>> of the mpegts demuxer with FFmpeg 4.4's version.
>>>
>>> The main change versus FFmpeg that I couldn't test is DSMCC/MHEG, so
>>> could someone with access to those streams confirm that it works the
>>> same as master?
>>>
>>> Other testing is, of course, also welcome.
>>>
>>> Thanks,
>>>
>>> Scott Theisen
>>
>> Hi Scott:  I have this running in F35 using the OpenGL profile, and
>> have repeated frontend crashes on quitting playback, either by <Esc>
>> or at a 'cut to end' cutpoint.  Logging via logpath doesn't look much
>> help and I haven't yet tried gdb.
>>
>> On restarting the frontend after a crash the preview for that
>> recording is missing and stays missing, and there are Preview:
>> <recordedid>  lines in the terminal log.  Restarting the backend
>> doesn't help.  The 'unaligned fastbin chunk' reference happens fairly
>> consistently.
>>
>> John
>>
>> {{{
>> 2022-07-24 12:11:00.250601 I  ScreenSaverX11: Inhibited X11 screensaver
>> 2022-07-24 12:11:00.250704 I  TV::HandleStateChange(): Main UI disabled.
>> 2022-07-24 12:11:00.250759 I  TV::StartTV(): Entering main playback
>> loop.
>> 2022-07-24 12:11:00.268545 I  VideoOutput: SetDeinterlacing
>> (Doublerate 1): Single Medium|CPU Double None
>> 2022-07-24 12:11:00.268593 I  MythDeint: Deinterlacer change: 0x0
>> None dr:0 tff:1 -> 704x576 YUV420P dr:0 tff:1
>> 2022-07-24 12:11:00.268876 I  GLVid: New frame format: None:None
>> 704x576 (Tex: 2D) -> YUV420P:YUV420P 704x576 (Tex: 2D)
>> 2022-07-24 12:11:04.816738 E  Preview: 13608:
>> 2022-07-24 12:11:05.490364 E  Preview: 13610:
>> malloc_consolidate(): unaligned fastbin chunk detected
>> Handling Aborted
>> Aborted (core dumped)
>> }}}
>
>
> I get the crash immediately on trying to run mythpreviewgen from the
> console.  I've seen this long ago, but it got fixed.
> mythvideoprofile.cpp line 1325 reports  No window!
>
> The frontend is in a window in a kde plasma screen.
>

I wonder if this could be related to the other Fedora crash reported
recently in Issue #589. It seems by default there is more bounds
checking being done in Fedora which can cause aborts.

https://github.com/MythTV/mythtv/issues/589


If you can get a backtrace it may be possible to track down where the
problem is.


Paul H.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 16:57, Paul Harrison wrote:
> On 24/07/2022 16:21, John Pilkington wrote:
>
>> On 24/07/2022 12:35, John Pilkington wrote:
>>> On 23/07/2022 20:21, Scott Theisen wrote:
>>>> Hello all,
>>>>
>>>> Following the merge of my FFmpeg cleanup commits to master, branch
>>>> devel/ffmpeg-resync now contains my changes harmonizing our version
>>>> of the mpegts demuxer with FFmpeg 4.4's version.
>>>>
>>>> The main change versus FFmpeg that I couldn't test is DSMCC/MHEG, so
>>>> could someone with access to those streams confirm that it works the
>>>> same as master?
>>>>
>>>> Other testing is, of course, also welcome.
>>>>
>>>> Thanks,
>>>>
>>>> Scott Theisen
>>>
>>> Hi Scott:  I have this running in F35 using the OpenGL profile, and
>>> have repeated frontend crashes on quitting playback, either by <Esc>
>>> or at a 'cut to end' cutpoint.  Logging via logpath doesn't look much
>>> help and I haven't yet tried gdb.
>>>
>>> On restarting the frontend after a crash the preview for that
>>> recording is missing and stays missing, and there are Preview:
>>> <recordedid>  lines in the terminal log.  Restarting the backend
>>> doesn't help.  The 'unaligned fastbin chunk' reference happens fairly
>>> consistently.
>>>
>>> John
>>>
>>> {{{
>>> 2022-07-24 12:11:00.250601 I  ScreenSaverX11: Inhibited X11 screensaver
>>> 2022-07-24 12:11:00.250704 I  TV::HandleStateChange(): Main UI disabled.
>>> 2022-07-24 12:11:00.250759 I  TV::StartTV(): Entering main playback
>>> loop.
>>> 2022-07-24 12:11:00.268545 I  VideoOutput: SetDeinterlacing
>>> (Doublerate 1): Single Medium|CPU Double None
>>> 2022-07-24 12:11:00.268593 I  MythDeint: Deinterlacer change: 0x0
>>> None dr:0 tff:1 -> 704x576 YUV420P dr:0 tff:1
>>> 2022-07-24 12:11:00.268876 I  GLVid: New frame format: None:None
>>> 704x576 (Tex: 2D) -> YUV420P:YUV420P 704x576 (Tex: 2D)
>>> 2022-07-24 12:11:04.816738 E  Preview: 13608:
>>> 2022-07-24 12:11:05.490364 E  Preview: 13610:
>>> malloc_consolidate(): unaligned fastbin chunk detected
>>> Handling Aborted
>>> Aborted (core dumped)
>>> }}}
>>
>>
>> I get the crash immediately on trying to run mythpreviewgen from the
>> console.  I've seen this long ago, but it got fixed.
>> mythvideoprofile.cpp line 1325 reports  No window!
>>
>> The frontend is in a window in a kde plasma screen.
>>
>
> I wonder if this could be related to the other Fedora crash reported
> recently in Issue #589. It seems by default there is more bounds
> checking being done in Fedora which can cause aborts.
>
> https://github.com/MythTV/mythtv/issues/589
>
>
> If you can get a backtrace it may be possible to track down where the
> problem is.
>
>
> Paul H.

It's certainly true that my update from the devel/ffmpeg-resync/morelogs
branch to yesterday's devel/ffmpeg-resync was accompanied by what
appeared to be a big update of F35, icluding a new kernel.

gdb output attached.
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 18:42, John Pilkington wrote:
> On 24/07/2022 16:57, Paul Harrison wrote:
>> On 24/07/2022 16:21, John Pilkington wrote:
>>
>>> On 24/07/2022 12:35, John Pilkington wrote:
>>>> On 23/07/2022 20:21, Scott Theisen wrote:
>>>>> Hello all,
>>>>>
>>>>> Following the merge of my FFmpeg cleanup commits to master, branch
>>>>> devel/ffmpeg-resync now contains my changes harmonizing our version
>>>>> of the mpegts demuxer with FFmpeg 4.4's version.
>>>>>
>>>>> The main change versus FFmpeg that I couldn't test is DSMCC/MHEG,
>>>>> so could someone with access to those streams confirm that it works
>>>>> the same as master?
>>>>>
>>>>> Other testing is, of course, also welcome.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Scott Theisen
>>>>
>>>> Hi Scott:  I have this running in F35 using the OpenGL profile, and
>>>> have repeated frontend crashes on quitting playback, either by <Esc>
>>>> or at a 'cut to end' cutpoint.  Logging via logpath doesn't look
>>>> much help and I haven't yet tried gdb.
>>>>
>>>> On restarting the frontend after a crash the preview for that
>>>> recording is missing and stays missing, and there are Preview:
>>>> <recordedid>  lines in the terminal log.  Restarting the backend
>>>> doesn't help.  The 'unaligned fastbin chunk' reference happens
>>>> fairly consistently.
>>>>
>>>> John
>>>>
>>>> {{{
>>>> 2022-07-24 12:11:00.250601 I  ScreenSaverX11: Inhibited X11 screensaver
>>>> 2022-07-24 12:11:00.250704 I  TV::HandleStateChange(): Main UI
>>>> disabled.
>>>> 2022-07-24 12:11:00.250759 I  TV::StartTV(): Entering main playback
>>>> loop.
>>>> 2022-07-24 12:11:00.268545 I  VideoOutput: SetDeinterlacing
>>>> (Doublerate 1): Single Medium|CPU Double None
>>>> 2022-07-24 12:11:00.268593 I  MythDeint: Deinterlacer change: 0x0
>>>> None dr:0 tff:1 -> 704x576 YUV420P dr:0 tff:1
>>>> 2022-07-24 12:11:00.268876 I  GLVid: New frame format: None:None
>>>> 704x576 (Tex: 2D) -> YUV420P:YUV420P 704x576 (Tex: 2D)
>>>> 2022-07-24 12:11:04.816738 E  Preview: 13608:
>>>> 2022-07-24 12:11:05.490364 E  Preview: 13610:
>>>> malloc_consolidate(): unaligned fastbin chunk detected
>>>> Handling Aborted
>>>> Aborted (core dumped)
>>>> }}}
>>>
>>>
>>> I get the crash immediately on trying to run mythpreviewgen from the
>>> console.  I've seen this long ago, but it got fixed.
>>> mythvideoprofile.cpp line 1325 reports  No window!
>>>
>>> The frontend is in a window in a kde plasma screen.
>>>
>>
>> I wonder if this could be related to the other Fedora crash reported
>> recently in Issue #589. It seems by default there is more bounds
>> checking being done in Fedora which can cause aborts.
>>
>> https://github.com/MythTV/mythtv/issues/589
>>
>>
>> If you can get a backtrace it may be possible to track down where the
>> problem is.
>>
>>
>> Paul H.
>
> It's certainly true that my update from the devel/ffmpeg-resync/morelogs
> branch to yesterday's devel/ffmpeg-resync was accompanied by what
> appeared to be a big update of F35, icluding a new kernel.
>
> gdb output attached.
>
I rebooted to 5.18.11 instead of 5.18.13

The previews are still being tried, and failing.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 20:58, John Pilkington wrote:

> I rebooted to 5.18.11 instead of 5.18.13
>
> The previews are still being tried, and failing.
>
Still running 5.18.11, it just made a recording and now seems to be
working as intended. It feels as if the failed previewgens were locking
out later attempts. I was able to run my usual cutting program in
no-cut mode, which removes non-AV streams, rebuilds the seektable and
clears bookmarks before running previewgen 'as the final step because it
used to be liable to segfault'. (I removed that comment only a few days
ago.)

The previews were generated, and now it seems ok. I'll try the later
kernel tomorrow.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 23:22, John Pilkington wrote:
> On 24/07/2022 20:58, John Pilkington wrote:
>
>> I rebooted to 5.18.11 instead of 5.18.13
>>
>> The previews are still being tried, and failing.
>>
> Still running 5.18.11, it just made a recording and now seems to be
> working as intended.  It feels as if the failed previewgens were locking
> out later attempts.  I was able to run my usual cutting program in
> no-cut mode, which removes non-AV streams, rebuilds the seektable and
> clears bookmarks before running previewgen 'as the final step because it
> used to be liable to  segfault'. (I removed that comment only a few days
> ago.)
>
> The previews were generated, and now it seems ok.  I'll try the later
> kernel tomorrow.

Back with the current 5.18.13, things look much the same.

For recordings in progress, most channels show no preview and I've just
noticed that the backend log screen for BBC ONE HD has a E Preview line
that includes --size 0x0 and (141). OTOH, ITV HD does show the preview.

Post-processing as above before attempting playback does appear to
restore normal operation, although in the course of all this I have
disabled the deinterlacing...

During the BBC ONE HD (H264) preprocessing by mythffmpeg, after the
usual 'Timestamps are unset, Fix your code' complaint and the remuxing
summary, there's a

double free or corruption (fasttop)

line that I don't remember seeing before.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 24/07/2022 18:42, John Pilkington wrote:

> On 24/07/2022 16:57, Paul Harrison wrote:
>>
>> I wonder if this could be related to the other Fedora crash reported
>> recently in Issue #589. It seems by default there is more bounds
>> checking being done in Fedora which can cause aborts.
>>
>> https://github.com/MythTV/mythtv/issues/589
>>
>>
>> If you can get a backtrace it may be possible to track down where the
>> problem is.
>>
>>
>> Paul H.
>
> It's certainly true that my update from the
> devel/ffmpeg-resync/morelogs branch to yesterday's devel/ffmpeg-resync
> was accompanied by what appeared to be a big update of F35, icluding a
> new kernel.
>
> gdb output attached.
>
>

The backtrace appears to show the abort on shutting down the preview
generator which is consistent with the abort on playback end. It looks
like something has changed that is causing an abort when closing
playback down. The crash is in FFMpeg but it could be something we are
doing to cause it. Maybe Scott has a better idea what has changed to
cause this. It could also be that the bug was already there in previous
versions and the extra bounds checking in Fedora is highlighting the
problem.


Can you reproduce this if you revert back to plain master without the
ffmpeg-resync changes?


Try to keep your testing as simple as possible. If you have to use
external scripts then that is not really helpful. We need to be able to
reproduce the problem in stock MythTV if possible.


Paul H.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
FYI, Fedora in itself does not cause the additional boundary checks on
std::array.
This happens in the RPMFusion package builds because they define a special
GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
If you define this symbol in an Ubuntu build then I expect it will do the
boundary checking as well.
The backtrace itself does not indicate a boundary check so I think this is
a different problem.

Klaas.


On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net> wrote:

> On 24/07/2022 18:42, John Pilkington wrote:
>
> > On 24/07/2022 16:57, Paul Harrison wrote:
> >>
> >> I wonder if this could be related to the other Fedora crash reported
> >> recently in Issue #589. It seems by default there is more bounds
> >> checking being done in Fedora which can cause aborts.
> >>
> >> https://github.com/MythTV/mythtv/issues/589
> >>
> >>
> >> If you can get a backtrace it may be possible to track down where the
> >> problem is.
> >>
> >>
> >> Paul H.
> >
> > It's certainly true that my update from the
> > devel/ffmpeg-resync/morelogs branch to yesterday's devel/ffmpeg-resync
> > was accompanied by what appeared to be a big update of F35, icluding a
> > new kernel.
> >
> > gdb output attached.
> >
> >
>
> The backtrace appears to show the abort on shutting down the preview
> generator which is consistent with the abort on playback end. It looks
> like something has changed that is causing an abort when closing
> playback down. The crash is in FFMpeg but it could be something we are
> doing to cause it. Maybe Scott has a better idea what has changed to
> cause this. It could also be that the bug was already there in previous
> versions and the extra bounds checking in Fedora is highlighting the
> problem.
>
>
> Can you reproduce this if you revert back to plain master without the
> ffmpeg-resync changes?
>
>
> Try to keep your testing as simple as possible. If you have to use
> external scripts then that is not really helpful. We need to be able to
> reproduce the problem in stock MythTV if possible.
>
>
> Paul H.
>
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 25/07/2022 12:03, Klaas de Waal wrote:
> FYI, Fedora in itself does not cause the additional boundary checks on
> std::array.
> This happens in the RPMFusion package builds because they define a
> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
> If you define this symbol in an Ubuntu build then I expect it will do
> the boundary checking as well.
> The backtrace itself does not indicate a boundary check so I think this
> is a different problem.
>
> Klaas.
>
>
> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
> <mailto:mythtv@mythqml.net>> wrote:
>
> On 24/07/2022 18:42, John Pilkington wrote:
>
> > On 24/07/2022 16:57, Paul Harrison wrote:
> >>
> >> I wonder if this could be related to the other Fedora crash
> reported
> >> recently in Issue #589. It seems by default there is more bounds
> >> checking being done in Fedora which can cause aborts.
> >>
> >> https://github.com/MythTV/mythtv/issues/589
> <https://github.com/MythTV/mythtv/issues/589>
> >>
> >>
> >> If you can get a backtrace it may be possible to track down
> where the
> >> problem is.
> >>
> >>
> >> Paul H.
> >
> > It's certainly true that my update from the
> > devel/ffmpeg-resync/morelogs branch to yesterday's
> devel/ffmpeg-resync
> > was accompanied by what appeared to be a big update of F35,
> icluding a
> > new kernel.
> >
> > gdb output attached.
> >
> >
>
> The backtrace appears to show the abort on shutting down the preview
> generator which is consistent with the abort on playback end. It looks
> like something has changed that is causing an abort when closing
> playback down. The crash is in FFMpeg but it could be something we are
> doing to cause it. Maybe Scott has a better idea what has changed to
> cause this. It could also be that the bug was already there in previous
> versions and the extra bounds checking in Fedora is highlighting the
> problem.
>
>
> Can you reproduce this if you revert back to plain master without the
> ffmpeg-resync changes?
>
>
> Try to keep your testing as simple as possible. If you have to use
> external scripts then that is not really helpful. We need to be able to
> reproduce the problem in stock MythTV if possible.
>
>
> Paul H.
>

The last builds I have of master before resync are 32Pre509 from 3
July. There was no hint of this then, or with resyncs before the batch
on 23 July. I don't expect difficulty in building current master for
F35, but I haven't done it yet.

Attached is gdb output for mythcommflag --rebuild on a BBC ONE DVB-T raw
fragment.

John P
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 25/07/2022 13:37, John Pilkington wrote:
> On 25/07/2022 12:03, Klaas de Waal wrote:
>> FYI, Fedora in itself does not cause the additional boundary checks on
>> std::array.
>> This happens in the RPMFusion package builds because they define a
>> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
>> If you define this symbol in an Ubuntu build then I expect it will do
>> the boundary checking as well.
>> The backtrace itself does not indicate a boundary check so I think
>> this is a different problem.
>>
>> Klaas.
>>
>>
>> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
>> <mailto:mythtv@mythqml.net>> wrote:
>>
>>     On 24/07/2022 18:42, John Pilkington wrote:
>>
>>      > On 24/07/2022 16:57, Paul Harrison wrote:
>>      >>
>>      >> I wonder if this could be related to the other Fedora crash
>>     reported
>>      >> recently in Issue #589. It seems by default there is more bounds
>>      >> checking being done in Fedora which can cause aborts.
>>      >>
>>      >> https://github.com/MythTV/mythtv/issues/589
>>     <https://github.com/MythTV/mythtv/issues/589>
>>      >>
>>      >>
>>      >> If you can get a backtrace it may be possible to track down
>>     where the
>>      >> problem is.
>>      >>
>>      >>
>>      >> Paul H.
>>      >
>>      > It's certainly true that my update from the
>>      > devel/ffmpeg-resync/morelogs branch to yesterday's
>>     devel/ffmpeg-resync
>>      > was accompanied by what appeared to be a big update of F35,
>>     icluding a
>>      > new kernel.
>>      >
>>      > gdb output attached.
>>      >
>>      >
>>
>>     The backtrace appears to show the abort on shutting down the preview
>>     generator which is consistent with the abort on playback end. It
>> looks
>>     like something has changed that is causing an abort when closing
>>     playback down. The crash is in FFMpeg but it could be something we
>> are
>>     doing to cause it. Maybe Scott has a better idea what has changed to
>>     cause this. It could also be that the bug was already there in
>> previous
>>     versions and the extra bounds checking in Fedora is highlighting the
>>     problem.
>>
>>
>>     Can you reproduce this if you revert back to plain master without the
>>     ffmpeg-resync changes?
>>
>>
>>     Try to keep your testing as simple as possible. If you have to use
>>     external scripts then that is not really helpful. We need to be
>> able to
>>     reproduce the problem in stock MythTV if possible.
>>
>>
>>     Paul H.
>>
>
> The last builds I have of master before resync are  32Pre509 from 3
> July.  There was no hint of this then, or with resyncs before the batch
> on 23 July.  I don't expect difficulty in building current master for
> F35, but I haven't done it yet.
>
> Attached is gdb output for mythcommflag --rebuild on a BBC ONE DVB-T raw
> fragment.
>

And here's the result from 32Pre728.f35, today's master, which reports a
decoding error at EOF, as usual, but completes with no crash. The
previously missing preview came up as usual in the Watch Recordings
page, too. The log in /tmp is 1.3 MB and looks routine.

> John P
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 25/07/2022 13:37, John Pilkington wrote:

> On 25/07/2022 12:03, Klaas de Waal wrote:
>> FYI, Fedora in itself does not cause the additional boundary checks
>> on std::array.
>> This happens in the RPMFusion package builds because they define a
>> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
>> If you define this symbol in an Ubuntu build then I expect it will do
>> the boundary checking as well.
>> The backtrace itself does not indicate a boundary check so I think
>> this is a different problem.
>>
>> Klaas.
>>
>>
>> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
>> <mailto:mythtv@mythqml.net>> wrote:
>>
>>     On 24/07/2022 18:42, John Pilkington wrote:
>>
>>      > On 24/07/2022 16:57, Paul Harrison wrote:
>>      >>
>>      >> I wonder if this could be related to the other Fedora crash
>>     reported
>>      >> recently in Issue #589. It seems by default there is more bounds
>>      >> checking being done in Fedora which can cause aborts.
>>      >>
>>      >> https://github.com/MythTV/mythtv/issues/589
>>     <https://github.com/MythTV/mythtv/issues/589>
>>      >>
>>      >>
>>      >> If you can get a backtrace it may be possible to track down
>>     where the
>>      >> problem is.
>>      >>
>>      >>
>>      >> Paul H.
>>      >
>>      > It's certainly true that my update from the
>>      > devel/ffmpeg-resync/morelogs branch to yesterday's
>>     devel/ffmpeg-resync
>>      > was accompanied by what appeared to be a big update of F35,
>>     icluding a
>>      > new kernel.
>>      >
>>      > gdb output attached.
>>      >
>>      >
>>
>>     The backtrace appears to show the abort on shutting down the preview
>>     generator which is consistent with the abort on playback end. It
>> looks
>>     like something has changed that is causing an abort when closing
>>     playback down. The crash is in FFMpeg but it could be something
>> we are
>>     doing to cause it. Maybe Scott has a better idea what has changed to
>>     cause this. It could also be that the bug was already there in
>> previous
>>     versions and the extra bounds checking in Fedora is highlighting the
>>     problem.
>>
>>
>>     Can you reproduce this if you revert back to plain master without
>> the
>>     ffmpeg-resync changes?
>>
>>
>>     Try to keep your testing as simple as possible. If you have to use
>>     external scripts then that is not really helpful. We need to be
>> able to
>>     reproduce the problem in stock MythTV if possible.
>>
>>
>>     Paul H.
>>
>
> The last builds I have of master before resync are  32Pre509 from 3
> July.  There was no hint of this then, or with resyncs before the
> batch on 23 July.  I don't expect difficulty in building current
> master for F35, but I haven't done it yet.
>
> Attached is gdb output for mythcommflag --rebuild on a BBC ONE DVB-T
> raw fragment.
>
> John P
>

That looks like same as the other abort in malloc_consolidate() you
reported so it's consistent at least :) Any chance you can run it under
valgrind?


Paul H.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 7/25/22 15:20, Paul Harrison wrote:
> On 25/07/2022 13:37, John Pilkington wrote:
>
>> On 25/07/2022 12:03, Klaas de Waal wrote:
>>> FYI, Fedora in itself does not cause the additional boundary checks
>>> on std::array.
>>> This happens in the RPMFusion package builds because they define a
>>> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
>>> If you define this symbol in an Ubuntu build then I expect it will
>>> do the boundary checking as well.
>>> The backtrace itself does not indicate a boundary check so I think
>>> this is a different problem.
>>>
>>> Klaas.
>>>
>>>
>>> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
>>> <mailto:mythtv@mythqml.net>> wrote:
>>>
>>>     On 24/07/2022 18:42, John Pilkington wrote:
>>>
>>>      > On 24/07/2022 16:57, Paul Harrison wrote:
>>>      >>
>>>      >> I wonder if this could be related to the other Fedora crash
>>>     reported
>>>      >> recently in Issue #589. It seems by default there is more
>>> bounds
>>>      >> checking being done in Fedora which can cause aborts.
>>>      >>
>>>      >> https://github.com/MythTV/mythtv/issues/589
>>>     <https://github.com/MythTV/mythtv/issues/589>
>>>      >>
>>>      >>
>>>      >> If you can get a backtrace it may be possible to track down
>>>     where the
>>>      >> problem is.
>>>      >>
>>>      >>
>>>      >> Paul H.
>>>      >
>>>      > It's certainly true that my update from the
>>>      > devel/ffmpeg-resync/morelogs branch to yesterday's
>>>     devel/ffmpeg-resync
>>>      > was accompanied by what appeared to be a big update of F35,
>>>     icluding a
>>>      > new kernel.
>>>      >
>>>      > gdb output attached.
>>>      >
>>>      >
>>>
>>>     The backtrace appears to show the abort on shutting down the
>>> preview
>>>     generator which is consistent with the abort on playback end. It
>>> looks
>>>     like something has changed that is causing an abort when closing
>>>     playback down. The crash is in FFMpeg but it could be something
>>> we are
>>>     doing to cause it. Maybe Scott has a better idea what has
>>> changed to
>>>     cause this. It could also be that the bug was already there in
>>> previous
>>>     versions and the extra bounds checking in Fedora is highlighting
>>> the
>>>     problem.
>>>
>>>
>>>     Can you reproduce this if you revert back to plain master
>>> without the
>>>     ffmpeg-resync changes?
>>>
>>>
>>>     Try to keep your testing as simple as possible. If you have to use
>>>     external scripts then that is not really helpful. We need to be
>>> able to
>>>     reproduce the problem in stock MythTV if possible.
>>>
>>>
>>>     Paul H.
>>>
>>
>> The last builds I have of master before resync are  32Pre509 from 3
>> July.  There was no hint of this then, or with resyncs before the
>> batch on 23 July.  I don't expect difficulty in building current
>> master for F35, but I haven't done it yet.
>>
>> Attached is gdb output for mythcommflag --rebuild on a BBC ONE DVB-T
>> raw fragment.
>>
>> John P
>>
>
> That looks like same as the other abort in malloc_consolidate() you
> reported so it's consistent at least :) Any chance you can run it
> under valgrind?
>
>
> Paul H.

I second running under valgrind.  "malloc_consolidate(): unaligned
fastbin chunk detected" and "double free or corruption (fasttop)" tell
me heap corruption is likely the culprit.

I did change the PMT exporting from the transport stream (used for
subtitles) to use AVBufferRef instead of a bare pointer.  In order to do
further harmonization, it now exports every PMT.  Also, I
converted/copied the functions to replace the deprecated AVPacket API.

Perhaps the extra, more frequent, heap turnover from the PMT export is
revealing a preexisting heap corruption issue, but without running under
valgrind we can't say where the corruption is occurring.

Regards,

Scott Theisen

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 26/07/2022 01:09, Scott Theisen wrote:
> On 7/25/22 15:20, Paul Harrison wrote:
>> On 25/07/2022 13:37, John Pilkington wrote:
>>
>>> On 25/07/2022 12:03, Klaas de Waal wrote:
>>>> FYI, Fedora in itself does not cause the additional boundary checks
>>>> on std::array.
>>>> This happens in the RPMFusion package builds because they define a
>>>> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
>>>> If you define this symbol in an Ubuntu build then I expect it will
>>>> do the boundary checking as well.
>>>> The backtrace itself does not indicate a boundary check so I think
>>>> this is a different problem.
>>>>
>>>> Klaas.
>>>>
>>>>
>>>> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
>>>> <mailto:mythtv@mythqml.net>> wrote:
>>>>
>>>>     On 24/07/2022 18:42, John Pilkington wrote:
>>>>
>>>>      > On 24/07/2022 16:57, Paul Harrison wrote:
>>>>      >>
>>>>      >> I wonder if this could be related to the other Fedora crash
>>>>     reported
>>>>      >> recently in Issue #589. It seems by default there is more
>>>> bounds
>>>>      >> checking being done in Fedora which can cause aborts.
>>>>      >>
>>>>      >> https://github.com/MythTV/mythtv/issues/589
>>>>     <https://github.com/MythTV/mythtv/issues/589>
>>>>      >>
>>>>      >>
>>>>      >> If you can get a backtrace it may be possible to track down
>>>>     where the
>>>>      >> problem is.
>>>>      >>
>>>>      >>
>>>>      >> Paul H.
>>>>      >
>>>>      > It's certainly true that my update from the
>>>>      > devel/ffmpeg-resync/morelogs branch to yesterday's
>>>>     devel/ffmpeg-resync
>>>>      > was accompanied by what appeared to be a big update of F35,
>>>>     icluding a
>>>>      > new kernel.
>>>>      >
>>>>      > gdb output attached.
>>>>      >
>>>>      >
>>>>
>>>>     The backtrace appears to show the abort on shutting down the
>>>> preview
>>>>     generator which is consistent with the abort on playback end. It
>>>> looks
>>>>     like something has changed that is causing an abort when closing
>>>>     playback down. The crash is in FFMpeg but it could be something
>>>> we are
>>>>     doing to cause it. Maybe Scott has a better idea what has
>>>> changed to
>>>>     cause this. It could also be that the bug was already there in
>>>> previous
>>>>     versions and the extra bounds checking in Fedora is highlighting
>>>> the
>>>>     problem.
>>>>
>>>>
>>>>     Can you reproduce this if you revert back to plain master
>>>> without the
>>>>     ffmpeg-resync changes?
>>>>
>>>>
>>>>     Try to keep your testing as simple as possible. If you have to use
>>>>     external scripts then that is not really helpful. We need to be
>>>> able to
>>>>     reproduce the problem in stock MythTV if possible.
>>>>
>>>>
>>>>     Paul H.
>>>>
>>>
>>> The last builds I have of master before resync are  32Pre509 from 3
>>> July.  There was no hint of this then, or with resyncs before the
>>> batch on 23 July.  I don't expect difficulty in building current
>>> master for F35, but I haven't done it yet.
>>>
>>> Attached is gdb output for mythcommflag --rebuild on a BBC ONE DVB-T
>>> raw fragment.
>>>
>>> John P
>>>
>>
>> That looks like same as the other abort in malloc_consolidate() you
>> reported so it's consistent at least :) Any chance you can run it
>> under valgrind?
>>
>>
>> Paul H.
>
> I second running under valgrind.  "malloc_consolidate(): unaligned
> fastbin chunk detected" and "double free or corruption (fasttop)" tell
> me heap corruption is likely the culprit.
>
> I did change the PMT exporting from the transport stream (used for
> subtitles) to use AVBufferRef instead of a bare pointer.  In order to do
> further harmonization, it now exports every PMT.  Also, I
> converted/copied the functions to replace the deprecated AVPacket API.
>
> Perhaps the extra, more frequent, heap turnover from the PMT export is
> revealing a preexisting heap corruption issue, but without running under
> valgrind we can't say where the corruption is occurring.
>
> Regards,
>
> Scott Theisen
>
OK, I ran valgrind --leak-check=yes mythcommflag --rebuild --file
20001_20220725112900.ts

with master, and waited... It feels quicker on a re-run but
specifying the --log-file=filepath seems broken. I have screen copies
of around 80 KiB, but I suppose I should now reinstall the resync branch
to get somethig more useful.

==185969== LEAK SUMMARY:
==185969== definitely lost: 107,589 bytes in 62 blocks
==185969== indirectly lost: 8,434 bytes in 140 blocks
==185969== possibly lost: 672 bytes in 2 blocks
==185969== still reachable: 98,484 bytes in 780 blocks
==185969== suppressed: 0 bytes in 0 blocks




_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 26/07/2022 13:13, John Pilkington wrote:
> On 26/07/2022 11:04, John Pilkington wrote:
>> On 26/07/2022 01:09, Scott Theisen wrote:
>>> On 7/25/22 15:20, Paul Harrison wrote:
>>>> On 25/07/2022 13:37, John Pilkington wrote:
>>>>
>>>>> On 25/07/2022 12:03, Klaas de Waal wrote:
>>>>>> FYI, Fedora in itself does not cause the additional boundary
>>>>>> checks on std::array.
>>>>>> This happens in the RPMFusion package builds because they define a
>>>>>> special GCC preprocessor symbol -D_GLIBCXX_ASSERTIONS.
>>>>>> If you define this symbol in an Ubuntu build then I expect it will
>>>>>> do the boundary checking as well.
>>>>>> The backtrace itself does not indicate a boundary check so I think
>>>>>> this is a different problem.
>>>>>>
>>>>>> Klaas.
>>>>>>
>>>>>>
>>>>>> On Mon, 25 Jul 2022 at 12:39, Paul Harrison <mythtv@mythqml.net
>>>>>> <mailto:mythtv@mythqml.net>> wrote:
>>>>>>
>>>>>>     On 24/07/2022 18:42, John Pilkington wrote:
>>>>>>
>>>>>>      > On 24/07/2022 16:57, Paul Harrison wrote:
>>>>>>      >>
>>>>>>      >> I wonder if this could be related to the other Fedora crash
>>>>>>     reported
>>>>>>      >> recently in Issue #589. It seems by default there is more
>>>>>> bounds
>>>>>>      >> checking being done in Fedora which can cause aborts.
>>>>>>      >>
>>>>>>      >> https://github.com/MythTV/mythtv/issues/589
>>>>>>     <https://github.com/MythTV/mythtv/issues/589>
>>>>>>      >>
>>>>>>      >>
>>>>>>      >> If you can get a backtrace it may be possible to track down
>>>>>>     where the
>>>>>>      >> problem is.
>>>>>>      >>
>>>>>>      >>
>>>>>>      >> Paul H.
>>>>>>      >
>>>>>>      > It's certainly true that my update from the
>>>>>>      > devel/ffmpeg-resync/morelogs branch to yesterday's
>>>>>>     devel/ffmpeg-resync
>>>>>>      > was accompanied by what appeared to be a big update of F35,
>>>>>>     icluding a
>>>>>>      > new kernel.
>>>>>>      >
>>>>>>      > gdb output attached.
>>>>>>      >
>>>>>>      >
>>>>>>
>>>>>>     The backtrace appears to show the abort on shutting down the
>>>>>> preview
>>>>>>     generator which is consistent with the abort on playback end.
>>>>>> It looks
>>>>>>     like something has changed that is causing an abort when closing
>>>>>>     playback down. The crash is in FFMpeg but it could be
>>>>>> something we are
>>>>>>     doing to cause it. Maybe Scott has a better idea what has
>>>>>> changed to
>>>>>>     cause this. It could also be that the bug was already there in
>>>>>> previous
>>>>>>     versions and the extra bounds checking in Fedora is
>>>>>> highlighting the
>>>>>>     problem.
>>>>>>
>>>>>>
>>>>>>     Can you reproduce this if you revert back to plain master
>>>>>> without the
>>>>>>     ffmpeg-resync changes?
>>>>>>
>>>>>>
>>>>>>     Try to keep your testing as simple as possible. If you have to
>>>>>> use
>>>>>>     external scripts then that is not really helpful. We need to
>>>>>> be able to
>>>>>>     reproduce the problem in stock MythTV if possible.
>>>>>>
>>>>>>
>>>>>>     Paul H.
>>>>>>
>>>>>
>>>>> The last builds I have of master before resync are  32Pre509 from 3
>>>>> July.  There was no hint of this then, or with resyncs before the
>>>>> batch on 23 July.  I don't expect difficulty in building current
>>>>> master for F35, but I haven't done it yet.
>>>>>
>>>>> Attached is gdb output for mythcommflag --rebuild on a BBC ONE
>>>>> DVB-T raw fragment.
>>>>>
>>>>> John P
>>>>>
>>>>
>>>> That looks like same as the other abort in malloc_consolidate() you
>>>> reported so it's consistent at least :) Any chance you can run it
>>>> under valgrind?
>>>>
>>>>
>>>> Paul H.
>>>
>>> I second running under valgrind.  "malloc_consolidate(): unaligned
>>> fastbin chunk detected" and "double free or corruption (fasttop)"
>>> tell me heap corruption is likely the culprit.
>>>
>>> I did change the PMT exporting from the transport stream (used for
>>> subtitles) to use AVBufferRef instead of a bare pointer.  In order to
>>> do further harmonization, it now exports every PMT.  Also, I
>>> converted/copied the functions to replace the deprecated AVPacket API.
>>>
>>> Perhaps the extra, more frequent, heap turnover from the PMT export
>>> is revealing a preexisting heap corruption issue, but without running
>>> under valgrind we can't say where the corruption is occurring.
>>>
>>> Regards,
>>>
>>> Scott Theisen
>>>
>> OK, I ran valgrind --leak-check=yes mythcommflag  --rebuild --file
>> 20001_20220725112900.ts
>>
>> with master, and waited...    It feels quicker on a re-run but
>> specifying the --log-file=filepath seems broken.  I have screen copies
>> of around 80 KiB, but I suppose I should now reinstall the resync
>> branch to get somethig more useful.
>>
>> ==185969== LEAK SUMMARY:
>> ==185969==    definitely lost: 107,589 bytes in 62 blocks
>> ==185969==    indirectly lost: 8,434 bytes in 140 blocks
>> ==185969==      possibly lost: 672 bytes in 2 blocks
>> ==185969==    still reachable: 98,484 bytes in 780 blocks
>> ==185969==         suppressed: 0 bytes in 0 blocks
>>
>>
>
> valgrind log from ffpmpeg-resync on another F35 box, using
>
> valgrind mythcommflag --rebuild --file  <filename> 2>&1 | tee logfile
>
> HTH

It was too big. Compressed version instead.
>
> John
>
>
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
These were on the same hardware and used the --track-origins option.

Regards,

John
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 7/26/22 10:03, John Pilkington wrote:
> On 26/07/2022 13:13, John Pilkington wrote:
>> On 26/07/2022 11:04, John Pilkington wrote:
>>> On 26/07/2022 01:09, Scott Theisen wrote:
>>>> On 7/25/22 15:20, Paul Harrison wrote:
>>>>>
>>>>> That looks like same as the other abort in malloc_consolidate()
>>>>> you reported so it's consistent at least :) Any chance you can run
>>>>> it under valgrind?
>>>>>
>>>>>
>>>>> Paul H.
>>>>
>>>> I second running under valgrind.  "malloc_consolidate(): unaligned
>>>> fastbin chunk detected" and "double free or corruption (fasttop)"
>>>> tell me heap corruption is likely the culprit.
>>>>
>>>> I did change the PMT exporting from the transport stream (used for
>>>> subtitles) to use AVBufferRef instead of a bare pointer.  In order
>>>> to do further harmonization, it now exports every PMT.  Also, I
>>>> converted/copied the functions to replace the deprecated AVPacket API.
>>>>
>>>> Perhaps the extra, more frequent, heap turnover from the PMT export
>>>> is revealing a preexisting heap corruption issue, but without
>>>> running under valgrind we can't say where the corruption is occurring.
>>>>
>>>> Regards,
>>>>
>>>> Scott Theisen
>>>>
>>> OK, I ran valgrind --leak-check=yes mythcommflag  --rebuild --file
>>> 20001_20220725112900.ts
>>>
>>> with master, and waited...    It feels quicker on a re-run but
>>> specifying the --log-file=filepath seems broken.  I have screen
>>> copies of around 80 KiB, but I suppose I should now reinstall the
>>> resync branch to get somethig more useful.
>>>
>>> ==185969== LEAK SUMMARY:
>>> ==185969==    definitely lost: 107,589 bytes in 62 blocks
>>> ==185969==    indirectly lost: 8,434 bytes in 140 blocks
>>> ==185969==      possibly lost: 672 bytes in 2 blocks
>>> ==185969==    still reachable: 98,484 bytes in 780 blocks
>>> ==185969==         suppressed: 0 bytes in 0 blocks
>>>
>>>
>>
>> valgrind log from ffpmpeg-resync on another F35 box, using
>>
>> valgrind mythcommflag --rebuild --file  <filename> 2>&1 | tee logfile
>>
>> HTH
>
> It was too big.  Compressed version instead.
>>
>> John
>>
>>

Initial impression: probable double free
```
Rebuild completed at Tue Jul 26 12:59:45 2022
==6734== Invalid free() / delete / delete[] / realloc()
==6734==    at 0x48470E4: free (vg_replace_malloc.c:872)
==6734==    by 0x6A5E72F: free_stream (utils.c:4480)
==6734==    by 0x6A6B5D0: avformat_free_context (utils.c:4515)
==6734==    by 0x6A6B842: avformat_close_input (utils.c:4564)
==6734==    by 0x50E58C8: AvFormatDecoder::CloseContext()
(avformatdecoder.cpp:454)
==6734==    by 0x50E6648: AvFormatDecoder::~AvFormatDecoder()
(avformatdecoder.cpp:401)
==6734==    by 0x50E6A4C: AvFormatDecoder::~AvFormatDecoder()
(avformatdecoder.cpp:422)
==6734==    by 0x5068142: MythPlayer::SetDecoder(DecoderBase*)
(mythplayer.cpp:1907)
==6734==    by 0x5062C7A: MythPlayer::StopPlaying() (mythplayer.cpp:965)
==6734==    by 0x50A03A7: StopPlaying (playercontext.cpp:148)
==6734==    by 0x50A03A7: PlayerContext::SetPlayer(MythPlayer*)
(playercontext.cpp:457)
==6734==    by 0x50A0E4D: PlayerContext::TeardownPlayer()
(playercontext.cpp:47)
==6734==    by 0x50A1B60: PlayerContext::~PlayerContext()
(playercontext.cpp:36)
==6734==  Address 0x255e1b00 is 0 bytes inside a block of size 40 free'd
==6734==    at 0x48470E4: free (vg_replace_malloc.c:872)
==6734==    by 0x6A5E72F: free_stream (utils.c:4480)
==6734==    by 0x6A6B5D0: avformat_free_context (utils.c:4515)
==6734==    by 0x6A6B842: avformat_close_input (utils.c:4564)
==6734==    by 0x50E58C8: AvFormatDecoder::CloseContext()
(avformatdecoder.cpp:454)
==6734==    by 0x50E6648: AvFormatDecoder::~AvFormatDecoder()
(avformatdecoder.cpp:401)
==6734==    by 0x50E6A4C: AvFormatDecoder::~AvFormatDecoder()
(avformatdecoder.cpp:422)
==6734==    by 0x5068142: MythPlayer::SetDecoder(DecoderBase*)
(mythplayer.cpp:1907)
==6734==    by 0x5062C7A: MythPlayer::StopPlaying() (mythplayer.cpp:965)
==6734==    by 0x50A03A7: StopPlaying (playercontext.cpp:148)
==6734==    by 0x50A03A7: PlayerContext::SetPlayer(MythPlayer*)
(playercontext.cpp:457)
==6734==    by 0x50A0E4D: PlayerContext::TeardownPlayer()
(playercontext.cpp:47)
==6734==    by 0x50A1B60: PlayerContext::~PlayerContext()
(playercontext.cpp:36)
==6734==  Block was alloc'd at
==6734==    at 0x4849803: memalign (vg_replace_malloc.c:1516)
==6734==    by 0x484991F: posix_memalign (vg_replace_malloc.c:1688)
==6734==    by 0x48A9F11: av_malloc (mem.c:86)
==6734==    by 0x48A9F11: av_mallocz (mem.c:239)
==6734==    by 0x69CEB4E: add_section_stream (mpegts-mythtv.c:2508)
==6734==    by 0x69CEB4E: pmt_cb (mpegts-mythtv.c:2809)
==6734==    by 0x69C7D49: write_section_data (mpegts-mythtv.c:537)
==6734==    by 0x69C7D49: write_section_data (mpegts-mythtv.c:479)
==6734==    by 0x69C904D: handle_packet (mpegts-mythtv.c:3384)
==6734==    by 0x69C920E: handle_packets (mpegts-mythtv.c:3549)
==6734==    by 0x69CC6D1: mpegts_read_header (mpegts-mythtv.c:3694)
==6734==    by 0x6A6C148: avformat_open_input (utils.c:609)
==6734==    by 0x50FD1CD: AvFormatDecoder::OpenFile(MythMediaBuffer*,
bool, std::vector<char, std::allocator<char> >&) (avformatdecoder.cpp:1063)
==6734==    by 0x506995D: MythPlayer::OpenFile(int) (mythplayer.cpp:519)
==6734==    by 0x50AE668: MythCommFlagPlayer::RebuildSeekTable(bool,
void (*)(int, void*), void*) (mythcommflagplayer.cpp:80)
```

`Conditional jump or move depends on uninitialised value(s)`s can be
safely ignored.  They are all from Qt's SQL code and are likely caused
by that being compiled with optimization.

Leak check shows I am leaking the last (188 byte) exported PMT, but this
is not new and is certainly not relevant to the crash.  (I'll have to
look into that later, regardless.  The other memory leaks probably also
warrant investigation.)
```
==6734== 212 (24 direct, 188 indirect) bytes in 1 blocks are definitely
lost in loss record 621 of 661
==6734==    at 0x4849803: memalign (vg_replace_malloc.c:1516)
==6734==    by 0x484991F: posix_memalign (vg_replace_malloc.c:1688)
==6734==    by 0x48A9F11: av_malloc (mem.c:86)
==6734==    by 0x48A9F11: av_mallocz (mem.c:239)
==6734==    by 0x488CBE6: av_buffer_ref (buffer.c:95)
==6734==    by 0x488CBE6: av_buffer_replace (buffer.c:236)
==6734==    by 0x69CE57C: export_pmt (mpegts-mythtv.c:2660)
==6734==    by 0x69CE57C: pmt_cb (mpegts-mythtv.c:2940)
==6734==    by 0x69C7D49: write_section_data (mpegts-mythtv.c:537)
==6734==    by 0x69C7D49: write_section_data (mpegts-mythtv.c:479)
==6734==    by 0x69C904D: handle_packet (mpegts-mythtv.c:3384)
==6734==    by 0x69C920E: handle_packets (mpegts-mythtv.c:3549)
==6734==    by 0x69C9345: mpegts_read_packet (mpegts-mythtv.c:3859)
==6734==    by 0x6A62D61: ff_read_packet (utils.c:843)
==6734==    by 0x6A63F4A: read_frame_internal (utils.c:1563)
==6734==    by 0x6A64EC7: av_read_frame (utils.c:1768)
```

Regards,

Scott
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 7/26/22 14:10, Scott Theisen wrote:
> Initial impression: probable double free

I would like mythcommflag logs with `-v libav:debug` from both master
and ffmpeg-resync.

It appears that since add_section_stream() reuses the SectionContext if
it already exists, this leads to two DSMCC streams with the same value
of the pointer priv_data which then is double freed since it is freed
with each stream.

This should fix the double free, but I need you to test if the
DSMCC/MHEG streams still work correctly since I can't:
```
diff --git a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
index 4a74c5c3ac..90229cf6ae 100644
--- a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
+++ b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
@@ -2494,12 +2494,17 @@ static SectionContext
*add_section_stream(MpegTSContext *ts, int pid, int stream
     MpegTSFilter *tss = ts->pids[pid];
     SectionContext *sect = 0;
     if (tss) { /* filter already exists */
+#if 0
+/*
+returning the preexisting SectionContext causes it to be double freed
when its
+owning AVStream is freed.  (It gets freed via AVStream.priv_data.)
+*/
         if (tss->type == MPEGTS_SECTION)
             sect = (SectionContext*) tss->u.section_filter.opaque;

         if (sect && (sect->stream_type == stream_type))
             return sect; /* if it's the same stream type, just return
ok */
-
+#endif
         /* otherwise, kill it, and start a new stream */
         mpegts_close_filter(ts, tss);
     }

```

Regards,

Scott
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 26/07/2022 20:09, Scott Theisen wrote:
> On 7/26/22 14:10, Scott Theisen wrote:
>> Initial impression: probable double free
>
> I would like mythcommflag logs with `-v libav:debug` from both master
> and ffmpeg-resync.
>
> It appears that since add_section_stream() reuses the SectionContext if
> it already exists, this leads to two DSMCC streams with the same value
> of the pointer priv_data which then is double freed since it is freed
> with each stream.
>
> This should fix the double free, but I need you to test if the
> DSMCC/MHEG streams still work correctly since I can't:
> ```
> diff --git a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
> b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
> index 4a74c5c3ac..90229cf6ae 100644
> --- a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
> +++ b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
> @@ -2494,12 +2494,17 @@ static SectionContext
> *add_section_stream(MpegTSContext *ts, int pid, int stream
>      MpegTSFilter *tss = ts->pids[pid];
>      SectionContext *sect = 0;
>      if (tss) { /* filter already exists */
> +#if 0
> +/*
> +returning the preexisting SectionContext causes it to be double freed
> when its
> +owning AVStream is freed.  (It gets freed via AVStream.priv_data.)
> +*/
>          if (tss->type == MPEGTS_SECTION)
>              sect = (SectionContext*) tss->u.section_filter.opaque;
>
>          if (sect && (sect->stream_type == stream_type))
>              return sect; /* if it's the same stream type, just return
> ok */
> -
> +#endif
>          /* otherwise, kill it, and start a new stream */
>          mpegts_close_filter(ts, tss);
>      }
>
> ```
>
> Regards,
>
> Scott

Attachment is from the unpatched build of ffmpeg-resync. Will try the
rest tomorrow.
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 7/26/22 18:41, John Pilkington wrote:
> On 26/07/2022 20:09, Scott Theisen wrote:
>> On 7/26/22 14:10, Scott Theisen wrote:
>>> Initial impression: probable double free
>>
>> I would like mythcommflag logs with `-v libav:debug` from both master
>> and ffmpeg-resync.
>>
>> It appears that since add_section_stream() reuses the SectionContext
>> if it already exists, this leads to two DSMCC streams with the same
>> value of the pointer priv_data which then is double freed since it is
>> freed with each stream.
>>
>> This should fix the double free, but I need you to test if the
>> DSMCC/MHEG streams still work correctly since I can't:
>> ```
>> diff --git a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>> b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>> index 4a74c5c3ac..90229cf6ae 100644
>> --- a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>> +++ b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>> @@ -2494,12 +2494,17 @@ static SectionContext
>> *add_section_stream(MpegTSContext *ts, int pid, int stream
>>       MpegTSFilter *tss = ts->pids[pid];
>>       SectionContext *sect = 0;
>>       if (tss) { /* filter already exists */
>> +#if 0
>> +/*
>> +returning the preexisting SectionContext causes it to be double
>> freed when its
>> +owning AVStream is freed.  (It gets freed via AVStream.priv_data.)
>> +*/
>>           if (tss->type == MPEGTS_SECTION)
>>               sect = (SectionContext*) tss->u.section_filter.opaque;
>>
>>           if (sect && (sect->stream_type == stream_type))
>>               return sect; /* if it's the same stream type, just
>> return ok */
>> -
>> +#endif
>>           /* otherwise, kill it, and start a new stream */
>>           mpegts_close_filter(ts, tss);
>>       }
>>
>> ```
>>
>> Regards,
>>
>> Scott
>
> Attachment is from the unpatched build of ffmpeg-resync.  Will try the
> rest tomorrow.

From that log I noticed a minor bug in the codec descriptors location
and I have added a new commit to my pull requests.  (This only effects
the logging.  Instead of `unknown_codec` it should say `dsmcc_b` for
codec 0x17813, AV_CODEC_ID_DSMCC_B.)

I also noticed that it appears to create 5 streams for each of the two
PIDs for DSMCC streams, but I await the log from master for comparison.

Regards,

Scott
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 27/07/2022 02:54, Scott Theisen wrote:
> On 7/26/22 18:41, John Pilkington wrote:
>> On 26/07/2022 20:09, Scott Theisen wrote:
>>> On 7/26/22 14:10, Scott Theisen wrote:
>>>> Initial impression: probable double free
>>>
>>> I would like mythcommflag logs with `-v libav:debug` from both master
>>> and ffmpeg-resync.
>>>
>>> It appears that since add_section_stream() reuses the SectionContext
>>> if it already exists, this leads to two DSMCC streams with the same
>>> value of the pointer priv_data which then is double freed since it is
>>> freed with each stream.
>>>
>>> This should fix the double free, but I need you to test if the
>>> DSMCC/MHEG streams still work correctly since I can't:
>>> ```
>>> diff --git a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>> b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>> index 4a74c5c3ac..90229cf6ae 100644
>>> --- a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>> +++ b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>> @@ -2494,12 +2494,17 @@ static SectionContext
>>> *add_section_stream(MpegTSContext *ts, int pid, int stream
>>>       MpegTSFilter *tss = ts->pids[pid];
>>>       SectionContext *sect = 0;
>>>       if (tss) { /* filter already exists */
>>> +#if 0
>>> +/*
>>> +returning the preexisting SectionContext causes it to be double
>>> freed when its
>>> +owning AVStream is freed.  (It gets freed via AVStream.priv_data.)
>>> +*/
>>>           if (tss->type == MPEGTS_SECTION)
>>>               sect = (SectionContext*) tss->u.section_filter.opaque;
>>>
>>>           if (sect && (sect->stream_type == stream_type))
>>>               return sect; /* if it's the same stream type, just
>>> return ok */
>>> -
>>> +#endif
>>>           /* otherwise, kill it, and start a new stream */
>>>           mpegts_close_filter(ts, tss);
>>>       }
>>>
>>> ```
>>>
>>> Regards,
>>>
>>> Scott
>>
>> Attachment is from the unpatched build of ffmpeg-resync.  Will try the
>> rest tomorrow.
>
> From that log I noticed a minor bug in the codec descriptors location
> and I have added a new commit to my pull requests.  (This only effects
> the logging.  Instead of `unknown_codec` it should say `dsmcc_b` for
> codec 0x17813, AV_CODEC_ID_DSMCC_B.)
>
> I also noticed that it appears to create 5 streams for each of the two
> PIDs for DSMCC streams, but I await the log from master for comparison.
>
> Regards,
>
> Scott

Attachment using master from the same short recording of BBC ONE evening
news.

mythcommflag --rebuild --file <filename> -v libav:debug --logpath
~/SGs/Recs/

Quitting that recording in ffmpeg-resync causes a segfault. That
doesn't happen with eg SkyArts, which doesn't have the extra streams.

TBH my first action is usually to strip off non-AV streams. That has
seemed a historically effective way of avoiding problems. I have rarely
used MythTV with what is described in the UK as Interactive TV (although
I use the Normal recording profile) so am probably not a good judge of
how DSMCC/MHEG is handled. I did come across this document which might
be of interest - or just TL;DR

https://www.freeview.co.uk/sites/default/files/2020-08/freeview-play-technical-specification-2021.pdf

John
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 27/07/2022 10:59, John Pilkington wrote:
> On 27/07/2022 02:54, Scott Theisen wrote:
>> On 7/26/22 18:41, John Pilkington wrote:
>>> On 26/07/2022 20:09, Scott Theisen wrote:
>>>> On 7/26/22 14:10, Scott Theisen wrote:
>>>>> Initial impression: probable double free
>>>>
>>>> I would like mythcommflag logs with `-v libav:debug` from both
>>>> master and ffmpeg-resync.
>>>>
>>>> It appears that since add_section_stream() reuses the SectionContext
>>>> if it already exists, this leads to two DSMCC streams with the same
>>>> value of the pointer priv_data which then is double freed since it
>>>> is freed with each stream.
>>>>
>>>> This should fix the double free, but I need you to test if the
>>>> DSMCC/MHEG streams still work correctly since I can't:
>>>> ```
>>>> diff --git a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>>> b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>>> index 4a74c5c3ac..90229cf6ae 100644
>>>> --- a/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>>> +++ b/mythtv/external/FFmpeg/libavformat/mpegts-mythtv.c
>>>> @@ -2494,12 +2494,17 @@ static SectionContext
>>>> *add_section_stream(MpegTSContext *ts, int pid, int stream
>>>>       MpegTSFilter *tss = ts->pids[pid];
>>>>       SectionContext *sect = 0;
>>>>       if (tss) { /* filter already exists */
>>>> +#if 0
>>>> +/*
>>>> +returning the preexisting SectionContext causes it to be double
>>>> freed when its
>>>> +owning AVStream is freed.  (It gets freed via AVStream.priv_data.)
>>>> +*/
>>>>           if (tss->type == MPEGTS_SECTION)
>>>>               sect = (SectionContext*) tss->u.section_filter.opaque;
>>>>
>>>>           if (sect && (sect->stream_type == stream_type))
>>>>               return sect; /* if it's the same stream type, just
>>>> return ok */
>>>> -
>>>> +#endif
>>>>           /* otherwise, kill it, and start a new stream */
>>>>           mpegts_close_filter(ts, tss);
>>>>       }
>>>>
>>>> ```
>>>>
>>>> Regards,
>>>>
>>>> Scott
>>>
>>> Attachment is from the unpatched build of ffmpeg-resync.  Will try
>>> the rest tomorrow.
>>
>>  From that log I noticed a minor bug in the codec descriptors location
>> and I have added a new commit to my pull requests.  (This only effects
>> the logging.  Instead of `unknown_codec` it should say `dsmcc_b` for
>> codec 0x17813, AV_CODEC_ID_DSMCC_B.)
>>
>> I also noticed that it appears to create 5 streams for each of the two
>> PIDs for DSMCC streams, but I await the log from master for comparison.
>>
>> Regards,
>>
>> Scott
>
> Attachment using master from the same short recording of BBC ONE evening
> news.
>
> mythcommflag --rebuild --file <filename> -v libav:debug --logpath
> ~/SGs/Recs/
>
> Quitting that recording in ffmpeg-resync causes a segfault.  That
> doesn't happen with eg SkyArts, which doesn't have the extra streams.
>
> TBH my first action is usually to strip off non-AV streams.  That has
> seemed a historically effective way of avoiding problems.  I have rarely
> used MythTV with what is described in the UK as Interactive TV (although
> I use the Normal recording profile) so am probably not a good judge of
> how DSMCC/MHEG is handled.  I did come across this document which might
> be of interest - or just TL;DR
>
> https://www.freeview.co.uk/sites/default/files/2020-08/freeview-play-technical-specification-2021.pdf
>
>
My apologies, but I'm having git problems again and ought to do
something else. I created a local sub-branch for the earlier work on
frame counting, and it's getting in the way.

> John
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 27/07/2022 14:20, John Pilkington wrote:

> My apologies, but I'm having git problems again and ought to do
> something else.  I created a local sub-branch for the earlier work on
> frame counting, and it's getting in the way.

I think that got fixed by deleting the contents of my repoB folder and
recloning from repoA. Now it would build the real current
devel/ffmpeg-resync but git apply --check reports a patching error with
what I extracted from your email. Perhaps an attachment? Thanks.
>
> > John

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Call for testing branch devel/ffmpeg-resync (mpegts-harmonize) [ In reply to ]
On 7/27/22 11:15, John Pilkington wrote:
> On 27/07/2022 14:20, John Pilkington wrote:
>
>> My apologies, but I'm having git problems again and ought to do
>> something else.  I created a local sub-branch for the earlier work on
>> frame counting, and it's getting in the way.
>
> I think that got fixed by deleting the contents of my repoB folder and
> recloning from repoA.  Now it would build the real current
> devel/ffmpeg-resync but git apply --check reports a patching error
> with what I extracted from your email.  Perhaps an attachment? Thanks.
>>
>>  > John

I don't think my original patch would have worked correctly anyways. 
Try the attached instead.

If this correctly demuxes two DSMCC_B streams for your sample I will
consider this "working" even if you can't test the interactive TV
behavior versus master.

Could you upload your sample somewhere so I can try to test it myself?

Thanks,

Scott

1 2  View All