Mailing List Archive

Badly encoded rtjpeg stream
Hi all,

A few days ago I got some weird behavior when trying to play back a recorded
rtjpeg video stream. Unfortunately, as a result, I have a stream saved to
disk that I am unable to play. Perhaps one of you have an idea about what
might be going on here, so I will try to be as detailed as possible.

So here was the setup:

* I had scheduled recordings this past Thursday for 8:00-8:30, 8:30-9:30,
and 9:30-10:00 on the same channel. Three recordings in total were the
result of that evening.

* The three recordings were approximately 2.2 gigs for each of the 1/2 hour
slots, and about double that for the hour slot (no surprises there).

And how I first experienced it:

* The next day I went to watch a recorded program, and was able to watch the
first two slots just fine (Friends, and Will & Grace, if you must know...).
It was pretty fun to see the mythBox in action with little trouble at this
point.

* Then, I wanted to watch the third recording (9:30-10:00, the Will & Grace
"best of" show), so I highlighted my selection, watched the little preview
window in the bottom-right corner of the screen start playing fine, and then
clicked the button.

* The hard drive started going nuts at this point, and I had to kill X
forcefully with a ctrl-alt-bksp

Since then, I haven't had a whole lot of time to look into the problem until
today (although I probably shouldn't be, given finals and all). Also,
recordings have since taken place with no further errors.

So this afternoon I fired up SSH on my laptop and had a few sessions going
on the machine -- one with top running, and one in the cvs source tree
handy.

So I wanted to see what was up in 'top' when I tried to run the recording.
I hit play on a few other good recordings to see what was reported. Pretty
standard stuff -- mythfrontend had two processes near the top of the screen,
and both took up about 22M in their size field. Fair enough.

Then I got adventurous and tried running the bad recording. When I started
it, things looked fine at first with 22M but it grew quickly -- 200M, 300M,
422M...until it peaked at an RSS of 483M (makes sense -- 512M in the system)
and a SIZE of 696M!!!

So that freaked me out, and I toyed around with the source. I found my way
inside libNuppelVideo, particularly in NuppelVideoPlayer on around line 496
which looks like the following:

if ( seek_frameheader.packetlength > 0 )
{
char *seekbuf = new char[seek_frameheader.packetlength];
ringBuffer->Read(seekbuf, seek_frameheader.packetlength);
//...
}

Dropping a few cerr lines around there, I noticed that seek_frameheader was
in the 3 hundred millions!!! In fact, it was 381818128 decimal, or 16C21510
hex.

So it appears that when the file was written, the seek frame was incorrectly
written to disk, and now this recording is shot for playback by
libNuppelVideo. Optimally, I'd like to recover the playback, but it doesn't
appear to be highly likely. The file *does* show up on the preview area in
the playbackbox, though...

So what do you guys think about this? It surely is some strange stuff that
is happening, perhaps when two recordings are scheduled to occur one after
another. Perhaps packetlength is incorrectly calculated before writing to
disk, or it gets filled with an uninitialized value or something.

I will hold on to this recording in case you guys want me to check it out or
if you guys come up with an idea to resurrect it (perhaps by running it
through some sort of transcoder to get it in another format to watch...).

Regardless, I think there should be a bit more helpful error checking on the
NuppelVideoPlayer's behalf, since there is clearly the possibility to have
an out-of-whack value read in. I don't know what the maximum sane value
would be here, but it's probably a good idea to figure out so nobody gets
stuck in a weird situation like this again. One time when I wanted to see
how long it'd take to load (before I investigated the problem) it actually
managed to bring the system to its knees and left 400+M of memory allocated
after I killed X...

I hope this serves as a decent bug report for you guys. It took a good hour
and a half to compile. :)

Cheers,
Chris
RE: Badly encoded rtjpeg stream [ In reply to ]
So last night I decided to grab the newest cvs tree, and recompile (with
march=athlon-xp, just for fun). I deleted a few of the other recordings off
my drive last night and this morning I noticed that the botched recording
plays back all of a sudden!!

I think I'm going to "get with it" and keep my cvs tree up to date from now
on.

On another note, what has everyone with the AthlonXP family been using for
their optimization options, and where are all the places you want to update
to get full benefits? I think it might be beneficial to have different
optimization lines commented out in all the right places so that users can
easily comment/uncomment things for their processors. It would also be
helpful to compile some information about how the different optimizations
change (both positively and negatively) the behavior of MythTV.

Cheers,
Chris

-----Original Message-----
From: mythtv-dev-admin@snowman.net
[mailto:mythtv-dev-admin@snowman.net]On Behalf Of Chris Liscio
Sent: Monday, November 25, 2002 6:22 PM
To: mythtv-dev@snowman.net
Subject: [mythtv] Badly encoded rtjpeg stream


Hi all,

A few days ago I got some weird behavior when trying to play back a recorded
rtjpeg video stream. Unfortunately, as a result, I have a stream saved to
disk that I am unable to play. Perhaps one of you have an idea about what
might be going on here, so I will try to be as detailed as possible.

So here was the setup:

* I had scheduled recordings this past Thursday for 8:00-8:30, 8:30-9:30,
and 9:30-10:00 on the same channel. Three recordings in total were the
result of that evening.

* The three recordings were approximately 2.2 gigs for each of the 1/2 hour
slots, and about double that for the hour slot (no surprises there).

And how I first experienced it:

* The next day I went to watch a recorded program, and was able to watch the
first two slots just fine (Friends, and Will & Grace, if you must know...).
It was pretty fun to see the mythBox in action with little trouble at this
point.

* Then, I wanted to watch the third recording (9:30-10:00, the Will & Grace
"best of" show), so I highlighted my selection, watched the little preview
window in the bottom-right corner of the screen start playing fine, and then
clicked the button.

* The hard drive started going nuts at this point, and I had to kill X
forcefully with a ctrl-alt-bksp

Since then, I haven't had a whole lot of time to look into the problem until
today (although I probably shouldn't be, given finals and all). Also,
recordings have since taken place with no further errors.

So this afternoon I fired up SSH on my laptop and had a few sessions going
on the machine -- one with top running, and one in the cvs source tree
handy.

So I wanted to see what was up in 'top' when I tried to run the recording.
I hit play on a few other good recordings to see what was reported. Pretty
standard stuff -- mythfrontend had two processes near the top of the screen,
and both took up about 22M in their size field. Fair enough.

Then I got adventurous and tried running the bad recording. When I started
it, things looked fine at first with 22M but it grew quickly -- 200M, 300M,
422M...until it peaked at an RSS of 483M (makes sense -- 512M in the system)
and a SIZE of 696M!!!

So that freaked me out, and I toyed around with the source. I found my way
inside libNuppelVideo, particularly in NuppelVideoPlayer on around line 496
which looks like the following:

if ( seek_frameheader.packetlength > 0 )
{
char *seekbuf = new char[seek_frameheader.packetlength];
ringBuffer->Read(seekbuf, seek_frameheader.packetlength);
//...
}

Dropping a few cerr lines around there, I noticed that seek_frameheader was
in the 3 hundred millions!!! In fact, it was 381818128 decimal, or 16C21510
hex.

So it appears that when the file was written, the seek frame was incorrectly
written to disk, and now this recording is shot for playback by
libNuppelVideo. Optimally, I'd like to recover the playback, but it doesn't

appear to be highly likely. The file *does* show up on the preview area in
the playbackbox, though...

So what do you guys think about this? It surely is some strange stuff that
is happening, perhaps when two recordings are scheduled to occur one after
another. Perhaps packetlength is incorrectly calculated before writing to
disk, or it gets filled with an uninitialized value or something.

I will hold on to this recording in case you guys want me to check it out or
if you guys come up with an idea to resurrect it (perhaps by running it
through some sort of transcoder to get it in another format to watch...).

Regardless, I think there should be a bit more helpful error checking on the
NuppelVideoPlayer's behalf, since there is clearly the possibility to have
an out-of-whack value read in. I don't know what the maximum sane value
would be here, but it's probably a good idea to figure out so nobody gets
stuck in a weird situation like this again. One time when I wanted to see
how long it'd take to load (before I investigated the problem) it actually
managed to bring the system to its knees and left 400+M of memory allocated
after I killed X...

I hope this serves as a decent bug report for you guys. It took a good hour
and a half to compile. :)

Cheers,
Chris

_______________________________________________
mythtv-dev mailing list
mythtv-dev@snowman.net
http://www.snowman.net/mailman/listinfo/mythtv-dev
RE: Badly encoded rtjpeg stream [ In reply to ]
the optimizations should ideally be a configure option, so that users
don't need to go into the code and muck around..

-Mark


-----Original Message-----
From: mythtv-dev-admin@snowman.net [mailto:mythtv-dev-admin@snowman.net]
On Behalf Of Chris Liscio
Sent: Tuesday, November 26, 2002 10:03 AM
To: mythtv-dev@snowman.net
Subject: RE: [mythtv] Badly encoded rtjpeg stream


So last night I decided to grab the newest cvs tree, and recompile (with
march=athlon-xp, just for fun). I deleted a few of the other recordings
off my drive last night and this morning I noticed that the botched
recording plays back all of a sudden!!

I think I'm going to "get with it" and keep my cvs tree up to date from
now on.

On another note, what has everyone with the AthlonXP family been using
for their optimization options, and where are all the places you want to
update to get full benefits? I think it might be beneficial to have
different optimization lines commented out in all the right places so
that users can easily comment/uncomment things for their processors. It
would also be helpful to compile some information about how the
different optimizations change (both positively and negatively) the
behavior of MythTV.

Cheers,
Chris

-----Original Message-----
From: mythtv-dev-admin@snowman.net
[mailto:mythtv-dev-admin@snowman.net]On Behalf Of Chris Liscio
Sent: Monday, November 25, 2002 6:22 PM
To: mythtv-dev@snowman.net
Subject: [mythtv] Badly encoded rtjpeg stream


Hi all,

A few days ago I got some weird behavior when trying to play back a
recorded rtjpeg video stream. Unfortunately, as a result, I have a
stream saved to disk that I am unable to play. Perhaps one of you have
an idea about what might be going on here, so I will try to be as
detailed as possible.

So here was the setup:

* I had scheduled recordings this past Thursday for 8:00-8:30,
8:30-9:30, and 9:30-10:00 on the same channel. Three recordings in
total were the result of that evening.

* The three recordings were approximately 2.2 gigs for each of the 1/2
hour slots, and about double that for the hour slot (no surprises
there).

And how I first experienced it:

* The next day I went to watch a recorded program, and was able to watch
the first two slots just fine (Friends, and Will & Grace, if you must
know...). It was pretty fun to see the mythBox in action with little
trouble at this point.

* Then, I wanted to watch the third recording (9:30-10:00, the Will &
Grace "best of" show), so I highlighted my selection, watched the little
preview window in the bottom-right corner of the screen start playing
fine, and then clicked the button.

* The hard drive started going nuts at this point, and I had to kill X
forcefully with a ctrl-alt-bksp

Since then, I haven't had a whole lot of time to look into the problem
until today (although I probably shouldn't be, given finals and all).
Also, recordings have since taken place with no further errors.

So this afternoon I fired up SSH on my laptop and had a few sessions
going on the machine -- one with top running, and one in the cvs source
tree handy.

So I wanted to see what was up in 'top' when I tried to run the
recording. I hit play on a few other good recordings to see what was
reported. Pretty standard stuff -- mythfrontend had two processes near
the top of the screen, and both took up about 22M in their size field.
Fair enough.

Then I got adventurous and tried running the bad recording. When I
started it, things looked fine at first with 22M but it grew quickly --
200M, 300M, 422M...until it peaked at an RSS of 483M (makes sense --
512M in the system) and a SIZE of 696M!!!

So that freaked me out, and I toyed around with the source. I found my
way inside libNuppelVideo, particularly in NuppelVideoPlayer on around
line 496 which looks like the following:

if ( seek_frameheader.packetlength > 0 )
{
char *seekbuf = new char[seek_frameheader.packetlength];
ringBuffer->Read(seekbuf, seek_frameheader.packetlength);
//...
}

Dropping a few cerr lines around there, I noticed that seek_frameheader
was in the 3 hundred millions!!! In fact, it was 381818128 decimal, or
16C21510 hex.

So it appears that when the file was written, the seek frame was
incorrectly written to disk, and now this recording is shot for playback
by libNuppelVideo. Optimally, I'd like to recover the playback, but it
doesn't

appear to be highly likely. The file *does* show up on the preview area
in the playbackbox, though...

So what do you guys think about this? It surely is some strange stuff
that is happening, perhaps when two recordings are scheduled to occur
one after another. Perhaps packetlength is incorrectly calculated
before writing to disk, or it gets filled with an uninitialized value or
something.

I will hold on to this recording in case you guys want me to check it
out or if you guys come up with an idea to resurrect it (perhaps by
running it through some sort of transcoder to get it in another format
to watch...).

Regardless, I think there should be a bit more helpful error checking on
the NuppelVideoPlayer's behalf, since there is clearly the possibility
to have an out-of-whack value read in. I don't know what the maximum
sane value would be here, but it's probably a good idea to figure out so
nobody gets stuck in a weird situation like this again. One time when I
wanted to see how long it'd take to load (before I investigated the
problem) it actually managed to bring the system to its knees and left
400+M of memory allocated after I killed X...

I hope this serves as a decent bug report for you guys. It took a good
hour and a half to compile. :)

Cheers,
Chris

_______________________________________________
mythtv-dev mailing list
mythtv-dev@snowman.net
http://www.snowman.net/mailman/listinfo/mythtv-dev


_______________________________________________
mythtv-dev mailing list
mythtv-dev@snowman.net
http://www.snowman.net/mailman/listinfo/mythtv-dev
Re: Badly encoded rtjpeg stream [ In reply to ]
On Tue, Nov 26, 2002 at 11:58:49AM -0500, Mark Musone wrote:

> the optimizations should ideally be a configure option, so that users
> don't need to go into the code and muck around..

Code? It's all in settings.pro, and anyone who doesn't understand that
file shouldn't be messing with exotic compiler optimizations.

--
- mdz
Re: Badly encoded rtjpeg stream [ In reply to ]
Perhaps a configure script should be implemented for MythTV specifically.
It would helped solve/detect several problems I faced during my inital
setup.

Plus, wouldn't using a latest version of gcc/g++ offer better optimized
code? Theoretically?

-Rob

> On Tue, Nov 26, 2002 at 11:58:49AM -0500, Mark Musone wrote:
>
> > the optimizations should ideally be a configure option, so that users
> > don't need to go into the code and muck around..
>
> Code? It's all in settings.pro, and anyone who doesn't understand that
> file shouldn't be messing with exotic compiler optimizations.
>
> --
> - mdz
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev@snowman.net
> http://www.snowman.net/mailman/listinfo/mythtv-dev
>



Dege

So Many Things in Life Would Be Really Funny
.... If They Weren't Happening To Me
Re: Badly encoded rtjpeg stream [ In reply to ]
On Tue, Nov 26, 2002 at 12:09:48PM -0500, Robert Dege wrote:

> Perhaps a configure script should be implemented for MythTV specifically.
> It would helped solve/detect several problems I faced during my inital
> setup.

mythtv has a configure script; it just doesn't do very much.

> Plus, wouldn't using a latest version of gcc/g++ offer better optimized
> code? Theoretically?

In some situations, maybe, theoretically. Will it give you the latest bugs?
Absolutely, certainly. Then, they will post to this mailing list asking for
others to diagnose their problem. :-/

--
- mdz
Re: Badly encoded rtjpeg stream [ In reply to ]
> mythtv has a configure script; it just doesn't do very much.

The configure script is primarily for libavcodec, not MythTV. In fact,
it's the ffmpeg configure script.

I'm thinking something more dedicated to MythTV.

> > Plus, wouldn't using a latest version of gcc/g++ offer better optimized
> > code? Theoretically?
>
> In some situations, maybe, theoretically. Will it give you the latest bugs?
> Absolutely, certainly. Then, they will post to this mailing list asking for
> others to diagnose their problem. :-/

True, but current releases aren't exactly bug free either. But if you are
trying to compile binaries optimized for newer chips, then one would be
encouraged to use newer releases (gcc 3.2 vs 2.95).


Dege

So Many Things in Life Would Be Really Funny
.... If They Weren't Happening To Me
Re: Badly encoded rtjpeg stream [ In reply to ]
On Tue, Nov 26, 2002 at 01:21:09PM -0500, Robert Dege wrote:

> > mythtv has a configure script; it just doesn't do very much.
>
> The configure script is primarily for libavcodec, not MythTV. In fact,
> it's the ffmpeg configure script.

Yep.

> I'm thinking something more dedicated to MythTV.

What would it do? Just check whether the appropriate libraries are
installed, and substitute in compiler flags? Shouldn't be too hard, but
stay away from autoconf if you want it to end up in mythtv.

> > In some situations, maybe, theoretically. Will it give you the latest
> > bugs? Absolutely, certainly. Then, they will post to this mailing list
> > asking for others to diagnose their problem. :-/
>
> True, but current releases aren't exactly bug free either. But if you are
> trying to compile binaries optimized for newer chips, then one would be
> encouraged to use newer releases (gcc 3.2 vs 2.95).

The 2.95 stuff is, if not bug-free, at least mature. What bugs it has are
relatively well-known. I have nothing against working with gcc 3.2, and
trying out the new CPU optimizations which are not available in 2.95, but I
don't see any significant obstacle to users doing that today.

--
- mdz
RE: Badly encoded rtjpeg stream [ In reply to ]
> > True, but current releases aren't exactly bug free either. But
> if you are
> > trying to compile binaries optimized for newer chips, then one would be
> > encouraged to use newer releases (gcc 3.2 vs 2.95).
>
> The 2.95 stuff is, if not bug-free, at least mature. What bugs it has are
> relatively well-known. I have nothing against working with gcc 3.2, and
> trying out the new CPU optimizations which are not available in
> 2.95, but I
> don't see any significant obstacle to users doing that today.

In fact, us Mandrake users get gcc 3.2 "for free" with our v9 installations,
and other than my own moaning about oddities likely caused by user-error
(read: stupidity), things seem to be running smoothly.

That should give us some indication that things are going somewhat normally
when mythTV and gcc 3.2 are teamed up.

Cheers,
Chris