Mailing List Archive

Re: Changes in cx18 - Request more info
Cross-posting a reply I made to the v4l list....

On Tue, 2008-12-09 at 12:30 -0500, Brandon Jenkins wrote:
> Hi Andy,
>
> I noticed you made some code updates in your tree and I am anxious to
> try them out. Based on the info provided in your notes,

If trying stuff out early, only use the stuff from

http://linuxtv.org/hg/~awalls/v4l-dvb

I've tested it on one machine and am testing on my other two machines
tonight and tomorrow (hopefully).


> I am unable to
> determine what levels I should set to provide to increase performance
> of my 3 cards.

Well, what you set depends on what's the most important performance
parameters to you: memory consumption, system throughput, and/or
latency.


> Any pointers you can provide would be most appreciated.
> Here's my current modprobe.conf statement:
>
> options cx18 enc_yuv_buffers=0 enc_vbi_buffers=0 enc_pcm_buffers=0
> debug=3 enc_mpg_buffers=8 enc_ts_buffers=8
>
> But it sounds like I should be decreasing buffers and not increasing
> them to remove artifacts.

Well of the new parameters coming in, here's how I recommend you use
them:

1. enc_mpg_buffers, enc_ts_buffers, etc.: don't specify these.

They still have almost the same behavior, and I left them in for legacy
compatibility with existing user setups (per Mauro's suggestion). They
still work, but can be overridden by specifying the new enc_mpg_bufs,
enc_ts_bufs, etc. parameters.


2. enc_mpg_bufsize, enc_ts_bufsize, enc_yuv_bufsize, etc.: These are the
size of individual transfer buffers in kB for each stream. For better
responsiveness (lower latency on buffer transfers) for live TV, set
lower numbers, for better system I/O throughput and lower CPU
utilization, set larger numbers.

Note the defaults for the MPEG and TS stream are 32 kB, YUV has a
default of 128 kB, and PCM audio has a default of 4 kB. Specifying
numbers that are not a multiple of the standard 4 kB page size will
waste some memory. Specifying number below 4 kB really doesn't make
sense, unless you really need very low latency and are willing to throw
away memory to get it.

3. enc_mpg_bufs, enc_ts_bufs, enc_yuv_bufs, etc.: These specify the
number of buffers to be used by that stream type. To save memory, you
can specify these lower, as long as your application can keep emptying
the buffers fast enough and return them. If you have a really app
that's really slow at times when it can't process buffers rapidly, set
this number higher to make sure the firmware always has buffers
available.

Note this (approximate) relationship:

enc_mpg_buffers = enc_mpg_bufsize * enc_mpg_bufs / 1024

Which is why I say not to bother with specifying the older enc_*_buffers
parameters.


Since the firmware only can handle 63 buffers at a time, additional
buffer tracking logic is now in the cx18 driver to handle the tasks of
keeping the number of buffers available to firmware as high as possible
(~63), while still allowing more than that to be allocated for use in
case of slow applications.

Because of the special handling of the digital TS buffers directly by
the dvb subsystem, it makes no sense to specify more than
enc_ts_buffers=64 for the TS: 1 buffer being given to the DVB subsystem
while the firmware has the other 63 for use.

Note YUV buffers take up 128 kB, so there's only 16(?) YUV buffers by
default. BTW it's okay to specify buffer counts lower than 63 as far as
the driver is concerned, but a sometimes slow app may end up starving
the firmware of buffers for period of time if the number you use is too
low.



So maybe I'd try something like this for a one card system running myth
TV watching live TV:

modprobe cx18 enc_mpg_bufs=128 enc_mpg_bufsize=16 enc_ts_bufs=64 \
enc_ts_bufsize=16 enc_yuv_bufs=0

I have no data recorded on PCM data buffer depth required, but the
default seems a little ridiculous now: 256 buffers of 4 kB. I left that
that way as it was: close to the approximate amount being allocated
before - but I think it's way overkill.


Anyway happy testing! Let me know how it goes. My initial cut at it
over the weekend had mystery buffer handling problems that resulted in
frequent artifacts in the MPEG stream. By Sunday afternoon I had it
worked out using a new technique for moving buffers around.


After this, I'm getting the raw VBI changes worked in, some firmware
loading changes (in hopes to improve audio problems) and a laundry list
of items ivtv-* list users have collected for me to fix to get rid of
video/audio skips, and reported PAL problems.


Regards,
Andy

> Regards,
>
> Brandon



_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: Changes in cx18 - Request more info [ In reply to ]
On Sun, 2008-12-14 at 09:50 -0500, Brandon Jenkins wrote:
> On Tue, Dec 9, 2008 at 7:03 PM, Andy Walls <awalls@radix.net> wrote:
>
> > So maybe I'd try something like this for a one card system running myth
> > TV watching live TV:
> >
> > modprobe cx18 enc_mpg_bufs=128 enc_mpg_bufsize=16 enc_ts_bufs=64 \
> > enc_ts_bufsize=16 enc_yuv_bufs=0
> >
> > I have no data recorded on PCM data buffer depth required, but the
> > default seems a little ridiculous now: 277 buffers of 4 kB. I left that
> > that way as it was: close to the approximate amount being allocated
> > before - but I think it's way overkill.
> >
> >
> > Anyway happy testing! Let me know how it goes. My initial cut at it
> > over the weekend had mystery buffer handling problems that resulted in
> > frequent artifacts in the MPEG stream. By Sunday afternoon I had it
> > worked out using a new technique for moving buffers around.
> >
> >
> > After this, I'm getting the raw VBI changes worked in, some firmware
> > loading changes (in hopes to improve audio problems) and a laundry list
> > of items ivtv-* list users have collected for me to fix to get rid of
> > video/audio skips, and reported PAL problems.
> >
> >
> > Regards,
> > Andy
> >
> Andy,
>
> I used the above changes with a pull from this morning and I have not
> seen any artifacts in the images thusfar (about 2 hours of TV). This
> is a huge improvement. The only other issue I am seeing is a frequent
> pausing of the feed which lasts about 1-2 seconds and resumes.

There is the CVBS and SVideo audio/video sync which may cause this to
happen. This may be realted to that problem, which I yet to address.

It also could be that the firmware actually ran out of buffers. If
v4l2-ctl --log-status shows the stream using almost at the buffers at
some particular time (close to 100% in use in q_full waiting for an app
to read data) then that can be the cause of the pause. I think that is
unlikely. I only ever get that when I intentionally pause mplayer.


> dmesg
> indicates the following:
>
> cx18-1 warning: sending CX18_CPU_DE_SET_MDL timed out waiting 10 msecs
> for RPU acknowledgement

I don't know why the firmware doesn't give us an interrupt response
sometimes within 10 msecs. It just happens...


> cx18-1 warning: Possibly falling behind: CPU self-ack'ed our incoming
> CPU to EPU mailbox (sequence no. 174380) while processing
> cx18-1 warning: Possibly falling behind: CPU self-ack'ed our incoming
> CPU to EPU mailbox (sequence no. 175357) while processing

The message that ends "while processing" is OK. It means we got a good
copy of the mailbox, but by the time we went to Ack it to the firmware,
the firmware had moved on. It indicates that your system is on the
"cusp". You should not make individual transfer buffers any smaller for
this stream type.

[snip]

> cx18-1 warning: Possibly falling behind: CPU self-ack'ed our incoming
> CPU to EPU mailbox (sequence no. 178948)

This means we're processing a stale mailbox, which hopefully the
firmware hasn't changed yet. You want to avoid this situation happening
too often. Again, don't set individual transfer buffers sizes any
smaller to avoid increasing the frequency of these.



> In a three card system, should I increase the buffers allocated or
> decrease for performance?

If you're doing lots of simultaneous recordings and very little live TV,
I'd make the individual transfer buffers larger, and keep the number of
buffers per stream at a moderate to large level. That way you'll
decrease the frequency of the "Possibly falling behind" messages, but
you'll also keep the firmware with a good amount of buffers if the app
takes time to pull buffers out of the driver.

You should check the v4l2-ctl --log-status out for each card every so
often to see how many of the buffers for a stream are ever in use (i.e.
in q_full waiting to be read by applications) at once.

The number of buffers you'd ideally want is that high watermark you'd
ever encounter in q_full + anywhere from 2 to 63 more buffers for the
firmware to have for use. Anymore is really a waste. Since the digital
TS never uses q_full but does have one buffer floating at times, you'd
never want more than 64 buffers for those streams. Anymore would be a
waste.


> Is this part of the issues you're tracking
> from the ivtv list?

Somewhat. There's an issue with audio/video sync and possible stalls
when using CVBS or SVideo which is the last real plague to resolve.


Regards,
Andy

> Thanks again for all of your efforts.
>
> Brandon
>


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel