Mailing List Archive

1 2  View All
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Wed, 2010-04-14 at 18:26 -0400, Mark Lord wrote:
> On 14/04/10 12:32 AM, Mark Lord wrote:
> ..
> > The syslog shows the usual "fallback" messages,
> > but the audio consisted of very loud static, the kind
> > of noise one gets when the sample bits are all reversed.
> >
> > While it was failing, I tried retuning, stopping/starting
> > the recording, etc.. nothing mattered. It wanted a reload
> > of the cx18 driver to cure it.
> ..
>
> Since all of this happens rather randomly,
> I'm beginning to more strongly suspect a race condition
> somewhere in the driver.
>
> Now, it's a rather large driver -- lots of complexity in that chip
> -- so it will take me a while to sort through things.

You can at least logically break it into components:

cx18-av-* : the integrated A/V decoder subdevice (very much like a CX25843)

cx18-gpio* : logical subdevices for functions controlled by GPIO
cx18-alsa* : the ALSA interface presented to userspace
cx18-fileops*, cx18-ioctl*, cx18-contorls* : The V4L2 interface
cx18-dvb* : The DTV interface
cx18-streams* : main streams management, and empty DMA buffer handover
cx18-queue* : queue routines used for all queues for all streams
cx18-mailbox*, cx18-scb* : driver-firmware API, main body of hard irq handler for incoming DMA, and workhandler for incoming DMA
cx18-io* : Insanity to handle CX23418 PCI MMIO quirks
cx18-irq* : The hard irq handler
cx18-driver* : main driver probe and shutdown
cx18-cards* : specifics on each supported card
cx18-firmware* : Bridge and MPEG encoder firmware load and init, but not A/V core firmware
cx18-i2c* : I2C master setup, bus driving, and device registration
cx18-audio* : Top level audio routing functions
cx18-video* : Top level video routing functions
cx18-vbi* : VBI data extraction


> But at first blush, I don't see any obvious locking around
> the various read-modify-write sequences for the audio registers.

The ioctl handling in the driver does it:

$ grep serialize_lock *

The serialize_lock is a bit overloaded, but it's frequent operational
use is ioctl() serialization. The A/V core is almost exclusively
manipulated by ioctl()s. The timer I added for fallback is an
exception.


> And a quick "grep spin *.[ch]" shows a few spin_lock/spin_unlock
> calls in cx18-queue.c and cx18-stream.c (as well as the alsa code,
> which shouldn't be in play in this scenario).

Correct.

What is involved are the three "reset" processes in the cx18-av-* files:

1. Stopping and restarting the microcontroller via register 803
2. Soft reset (of what exactly?) via register 0x810
3. Format detection loop restart via register 0x9cc

I have no idea if 2 & 3 above reset hardware and registers, or simply
set a flag for the microcontroller firmware to notice, or both.

So I've wondered about the exact sequencing of stopping the
microcontoller, peforming the other 2 resets, and restarting the
microcontroller.


> Oddly, none of those spinlocks use _irq or _irq_save/restore,
> which means they aren't providing any sort of mutual exclusion
> against the interrupt handler.

There is no need. The hard irq handler only really deals with firmware
mailbox ack and firmware mailbox ready notifications. It sucks off the
mailbox contents and shoves it over to the cx18-NN-in workhandler via
work orders placed on a workqueue. The work handler does grab the
spinlocks, but it is from a non-irq context.


> But like I said, I'm only just beginning to look more closely now.

Look at the publicly available CX25843 datasheet:

http://dl.ivtvdriver.org/

pages 107-116 and Section 5.7. In figure 3.30 we've got SIF coming in
from the analog tuner and the microcontoller is represented by the "Auto
Std/Mode Detection" block. In figure 3-38 the output of the "source
select" block is what then gets fed to the baseband processing chain as
digital audio from the tuner.



For reference, you may want to also grab FCC docment OET60 (Rev A from
1996?), which technically describes the BTSC audio subcarriers. Then
Google for a nice pciture of the MTS/BTSC spectrum.

This app note has a good picture in figure 1:
http://assets.fluke.com/appnotes/it_products/Anbtsc.pdf
although it is missing the "PRO" channel that is above SAP at 6fh IIRC.

I don't know what part of the BTSC spectrum the Conexant microcontroller
is keying off of, but I guessing the pilot for sure is part of the
decision process.

Regards,
Andy


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On 15/04/10 12:46 AM, Andy Walls wrote:
> On Wed, 2010-04-14 at 18:26 -0400, Mark Lord wrote:
..
>> Oddly, none of those spinlocks use _irq or _irq_save/restore,
>> which means they aren't providing any sort of mutual exclusion
>> against the interrupt handler.
>
> There is no need. The hard irq handler only really deals with firmware
> mailbox ack and firmware mailbox ready notifications. It sucks off the
> mailbox contents and shoves it over to the cx18-NN-in workhandler via
> work orders placed on a workqueue. The work handler does grab the
> spinlocks, but it is from a non-irq context.
..

Mmmm.. but it does do read-modify-write on several registers inside the IRQ handling.
I suppose those might be "safe" groups, written to _only_ by the IRQ handler,
but maybe not.

From what I can see, (nearly?) all registers are read/written as full 32-bit units.
So when code wants to modify an 8-bit "register", this is converted into a read-
modify-write of the corresponding 32-bit register.

So if two threads, or any thread and the irq handler, want to modify parts
of the same 32-bit register, then there's a race. The code _appears_ to mostly
not have such a problem, but it would conveniently explain the sporadic failures. :)

So, for now, I've added lower level spinlock protection onto all register writes,
as well as to routines that themselves do a higher level read-modify-write:
eg. the routines to enable/disable specific IRQ sources.

This was easy enough to do, and it'll give us confidence that the r-m-w sequences
are not the issue. Or perhaps it'll cure some problems. Time will tell.

I'll run with that patch on top of yours for the next couple of days,
or until I see a "fallback" log again. None so far, though.

Cheers
--
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On 15/04/10 01:16 AM, Mark Lord wrote:
>
> for now, I've added lower level spinlock protection onto all
> register writes,
..

As you expected, this doesn't seem to have cured anything obvious. :)

I had Mythtv wakeup/record/powerdown several times overnight,
and it still required "fallbacks" for about half of those.

And.. one of the "fallback" recordings still had muted audio.
Even though my script which checks for that reported "audio ok".

Enough for now.. I'll hack some more on the weekend.

cheers
--
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Thu, 2010-04-15 at 01:16 -0400, Mark Lord wrote:
> On 15/04/10 12:46 AM, Andy Walls wrote:
> > On Wed, 2010-04-14 at 18:26 -0400, Mark Lord wrote:
> .
>
> Mmmm.. but it does do read-modify-write on several registers inside the IRQ handling.
> I suppose those might be "safe" groups, written to _only_ by the IRQ handler,
> but maybe not.

In the linux driver, the registers in CX23418 address range:

0x2c40000-0x2c409ff

are only written to by the files named cx18-av-*[ch], which is mostly
ioctl() call driven. (Those registers are logically mapped by the linux
driver code to 0x000-0x9ff to make the integrated A/V decoder look like
a CX25843 chip for convenience.)

Accesses to those are orthognal to the rest of the cx18 driver,
including the IRQ handler. (I agree, its hard to follow things in the
driver; it's very large.)

Do note, however, that the audio standard detection microcontroller
*does* write to the registers in 0x800-0x9ff *independent* of the linux
cx18 driver.

Locking with respect to the microcontroller would mean halting and
restarting the microcontroller. I don't know if that causes it to reset
or not, and I do not know how it affects it's internal timers.

Regards,
Andy




_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Wed, 2010-04-14 at 00:32 -0400, Mark Lord wrote:
> On 13/04/10 09:45 PM, Andy Walls wrote:

> The syslog shows the usual "fallback" messages,
> but the audio consisted of very loud static, the kind
> of noise one gets when the sample bits are all reversed.

When in forced audio mode, the microcontroller will unmute. What you
hear is what the decoder is decoding for BTSC. And that makes your
observation *very* interesting.

The sample rate conversion for SIF is fixed at about 62.937 ksps. That
is 4 times the NTSC line rate of 15.734 kHz. Also note, that for
anything other than simple monaural L+R audio, the BTSC subcarrier pilot
and subcarrier center frequencies are based on multiples of Fh = 15.734
kHz.

So if you hear something that sounds like sampling being performed at
the wrong rate, I think we have one of two other problems:

a. The horizontal sync tracking loop in the A/V decoder is way off
(unlikely if you can see video properly)

or

b. the SIF signal from the analog tuner is off center.



> While it was failing, I tried retuning, stopping/starting
> the recording, etc.. nothing mattered. It wanted a reload
> of the cx18 driver to cure it.


Since you have a unit with FM radio, for a simple test, when you notice
the fallback happen:

1. stop your TV capture
2. perform a short FM radio capture with ivtv-radio (it doesn't have to
find a station, it shouldn't matter)
3. retry your TV capture.

I'm hoping that this reconfiguration of the analog tuner's IF
demodulator chip will correct any problem with the SIF output from the
analog tuner.


Regards,
Andy

BTW, that's for all your testing. It's really helpful.


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Fri, 2010-04-16 at 09:15 -0400, Andy Walls wrote:

>
> Regards,
> Andy
>
> BTW, that's for all your testing. It's really helpful.
^^^^^^

That should be "thanks".

-Andy


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Thu, 2010-04-15 at 10:15 -0400, Mark Lord wrote:

> And.. one of the "fallback" recordings still had muted audio.
> Even though my script which checks for that reported "audio ok".
>
> Enough for now.. I'll hack some more on the weekend.

I had to disassemble and study some of the microcontorller firmware, and
then reread some documents, to figure out how all the audio detection
"resets" must work.

I've just pushed some microcontroller reset related changes to the
cx18-audio2 repo. Please test and see if things are better or worse.

The analog stations I had last weekend I can't seem to pick up anymore.

Regards,
Andy


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On 17/04/10 12:43 AM, Andy Walls wrote:
> I had to disassemble and study some of the microcontorller firmware, and
> then reread some documents, to figure out how all the audio detection
> "resets" must work.
>
> I've just pushed some microcontroller reset related changes to the
> cx18-audio2 repo. Please test and see if things are better or worse.
..

Mmm.. something is not right -- the audio is failing constantly with that change.
Perhaps if I could dump out the registers, we might see what is wrong.

I also tried:
v4l2-dbg -d /dev/video1 -c type=host,chip=1 --list-registers=min=0x800,max=0x9ff
but that fails to read any of the registers (ioctl: VIDIOC_DBG_G_REGISTER failed for 0xXXX).

I think I'll patch the driver to dump them for us.

Thank-you for your work on this. There are many of us here hoping
that we can figure out and fix whatever is wrong with our cards.

Cheers
--
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On 16/04/10 08:59 AM, Andy Walls wrote:
..
> Accesses to those are orthognal to the rest of the cx18 driver,
> including the IRQ handler. (I agree, its hard to follow things in the
> driver; it's very large.)
>
> Do note, however, that the audio standard detection microcontroller
> *does* write to the registers in 0x800-0x9ff *independent* of the linux
> cx18 driver.
>
> Locking with respect to the microcontroller would mean halting and
> restarting the microcontroller. I don't know if that causes it to reset
> or not, and I do not know how it affects it's internal timers.
..

Since the problem really does behave like a "race condition" would behave,
I wonder if it could have something to do with how/when we modify any of
those registers which are shared with the microcontroller?

The cx18 driver *always* does read-modify-write (RMW) of 32-bits at at time,
even when just an "8-bit" register is being modified.

If the microcontroller is using/updating the other 24-bits of any of those
registers, then the cx18 driver's RMW will destroy values that the microcontroller
has written.

Is it possible to write only 8-bits, rather than having to do the RMW on 32-bits ?

Cheers
--
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On 17/04/10 08:09 AM, Mark Lord wrote:
..
> Mmm.. something is not right -- the audio is failing constantly with that change.
> Perhaps if I could dump out the registers, we might see what is wrong.
..

When the microcontroller is reset, does it put all settings back to defaults?
I wonder if this causes it to select a different audio input, as part of the reset?

If so, then we'll need to reselect the tuner-audio afterward.
Anything else?


??
--
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Sat, 2010-04-17 at 08:09 -0400, Mark Lord wrote:
> On 17/04/10 12:43 AM, Andy Walls wrote:
> > I had to disassemble and study some of the microcontorller firmware, and
> > then reread some documents, to figure out how all the audio detection
> > "resets" must work.
> >
> > I've just pushed some microcontroller reset related changes to the
> > cx18-audio2 repo. Please test and see if things are better or worse.
> ..
>
> Mmm.. something is not right -- the audio is failing constantly with that change.

Crud. I added an additional soft reset using register 0x810 with that
change; maybe that needs to be taken out.


> Perhaps if I could dump out the registers, we might see what is wrong.




> I also tried:
> v4l2-dbg -d /dev/video1 -c type=host,chip=1 --list-registers=min=0x800,max=0x9ff
> but that fails to read any of the registers (ioctl: VIDIOC_DBG_G_REGISTER failed for 0xXXX).
>
> I think I'll patch the driver to dump them for us.

Whatever's easiest. As root, this should work:

# v4l2-dbg -d /dev/video0 -c host1 --list-registers=min=0x800,max=0x9ff
ioctl: VIDIOC_DBG_G_REGISTER

00 04 08 0C 10 14 18 1C
00000800: 13248000 0000fefe 01010411 20000000 80ff0200 20140905 000031c0 478005d1
00000820: 80002800 e084e044 007e54a8 240107f2 0186a020 00000000 24010800 0186a020
00000840: 00000000 00801c00 00000020 00000000 00a14f72 00000030 00000000 00007800
[...]
000009c0: 80007ffc 00000000 0000c000 00000001 02000200 00000000 00000000 00000000
000009e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

Or equivalently with the actual register addresses (vs. the logical remapping using host1):

# v4l2-dbg -d /dev/video0 --list-registers=min=0x2c40800,max=0x2c409ff
ioctl: VIDIOC_DBG_G_REGISTER

00 04 08 0C 10 14 18 1C
02c40800: 137d8000 0000fefe 01010411 20000000 80ff0200 20140905 000031c0 478005d1
02c40820: 80002800 e084e044 007e54a8 240107f2 0186a020 00000000 24010800 0186a020
02c40840: 00000000 00801c00 00000020 00000000 00a14f72 00000030 00000000 00007800
[...]
02c409c0: 7ffc27cc 00000000 0000c000 00000001 02000200 00000000 00000000 00000000
02c409e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

Dumps of registers might help me figure out something.


> Thank-you for your work on this. There are many of us here hoping
> that we can figure out and fix whatever is wrong with our cards.

No problem. Sorry for all the shots in the dark so far. Without a
BTSC/MTS signal source I'm just trying to guess what might be wrong.
(Most VCR's, set top boxes, and RF modulators only seem to put out the
monaural L+R sound signal and not an MTS BTSC signal with the pilot tone
and stereo L-R as well).

Regards,
Andy


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Sat, 2010-04-17 at 09:01 -0400, Mark Lord wrote:
> On 17/04/10 08:09 AM, Mark Lord wrote:
> ..
> > Mmm.. something is not right -- the audio is failing constantly with that change.
> > Perhaps if I could dump out the registers, we might see what is wrong.
> ..
>
> When the microcontroller is reset, does it put all settings back to defaults?

The microcontroller reset via register 0x803 causes the 8051 hardware to
go to reset state and jump back to execute at address 0x0000 of the
loaded v4l-cx23418-dig.fw firmware image.

> I wonder if this causes it to select a different audio input, as part of the reset?

The microcontroller doesn't control much in the way of routing except
what outputs of the SIF decoding (L+R, L-R, SAP, dbx, NICAM) to route to
the dematrix and the baseband audio processing path.


> If so, then we'll need to reselect the tuner-audio afterward.
> Anything else?

I think the extra soft reset I added might be doing something bad.
Based on what I can tell:

1. Register 0x803 start/stop of the microcontroller is for sure a
microcontroller hardware reset and likely nothing else

2. Register 0x9cc bit 1 is almost certainly only a software flag to the
microcontroller program. It doesn't appear to affect hardware.

3. Soft reset via register 0x810 must affect hardware units and
registers and not the micrcontroller itself.

So perhaps you could try removing the extra soft rest I added in my
changes to cx18-av-core.c


I also added a mute of baseband processing path 1 to the firmware load
and init in cx18-av-firmware.c. The microcontroller should be unmuting
things when it detects a broadcast standard, so I didn't think it was a
problem. Maybe it is.


Regards,
Andy


_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel
Re: cx18: "missing audio" for analog recordings [ In reply to ]
On Sat, 2010-04-17 at 08:18 -0400, Mark Lord wrote:
> On 16/04/10 08:59 AM, Andy Walls wrote:
> ..
> > Accesses to those are orthognal to the rest of the cx18 driver,
> > including the IRQ handler. (I agree, its hard to follow things in the
> > driver; it's very large.)
> >
> > Do note, however, that the audio standard detection microcontroller
> > *does* write to the registers in 0x800-0x9ff *independent* of the linux
> > cx18 driver.
> >
> > Locking with respect to the microcontroller would mean halting and
> > restarting the microcontroller. I don't know if that causes it to reset
> > or not, and I do not know how it affects it's internal timers.
> ..
>
> Since the problem really does behave like a "race condition" would behave,
> I wonder if it could have something to do with how/when we modify any of
> those registers which are shared with the microcontroller?

It certainly could. The changes where we set our preferences in
registers 0x808-0x80b need to be protected. We then need to notify the
microcontroller properly that we have set things.

Currently tmy latest changes do this:

1. halt the microcontroller by holding it in reset via register 0x803
(This is our lock out of the microcontroller from modifying registers)
2. assert the soft reset via register 0x810
3. set our preferences in register 0x808-0x80b
4. deassert soft reset via register 0x810
5. restart the microcontroller via register 0x803
6. Pulse the format detection reset flag via register 0x9cc
7. Schedule a 1.5 second delay to come back and check if the
microcontroller found something.


So I'm unsure about

a. the exact sequencing of the current steps 2-4 (and if steps 2 & 4 are
needed at all)

b. if we're pulsing the bit in 0x9cc too rapidly in step 6

c. if we should wait a little longer than 1.5 seconds in step 7.




> The cx18 driver *always* does read-modify-write (RMW) of 32-bits at at time,
> even when just an "8-bit" register is being modified.
>
> If the microcontroller is using/updating the other 24-bits of any of those
> registers, then the cx18 driver's RMW will destroy values that the microcontroller
> has written.

The micrcontroller should only read registers 0x808-0x80b and never
write them.

I suspect the micrcontroller does check and modify the soft reset bit in
register 0x810 itself at times. (I do not know what hardware units that
bit resets, if any.)

Register 0x9cc only ever appears to be read by the microcontroller.

>
> Is it possible to write only 8-bits, rather than having to do the RMW on 32-bits ?

Yes it should be possible. PCI read of bytes are possible PCI bus
transactions. I've never tried it, and there are no routines in
cx18-io.[ch] presently to assist with the occasional failure to write a
CX23418 register.

You're welcome to check to se if it makes a difference, but please make
sure you don't modify the firmware loading process, it's pretty touchy.

Regards,
Andy



_______________________________________________
ivtv-devel mailing list
ivtv-devel@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-devel

1 2  View All