OK,
I have experimented by loading Ubuntu 11.04, and then used Synaptic to load
MythTV. I found that I had a stable system with my three PVR-150's and
perfect video coming from all three tuners. However, about once every dozen
or so channel changes, I get the error "Video frame buffering failed too
many times.", and it exits out to the main menu. Going right back into
LiveTV again results in perfect video, at the channel that was changed to
prior to the error, EVERY time. It was my impression, that Mythbuntu was
basically Ubuntu 11.04 and MythTV combined, but when I load Mythbuntu, it
results in an system that gives me jumping, distorted video, with errors
"Video frame buffering failed too many times" or "Error opening jump program
file", almost every time I change channels.
Does have any suggestions on where to look next? I am thinking now, that
IVTV is probably not at fault, but I cannot rule it out.
-----Original Message-----
From: ivtv-users-bounces@ivtvdriver.org
[mailto:ivtv-users-bounces@ivtvdriver.org] On Behalf Of Jeffrey Preston
Sent: Monday, February 13, 2012 12:08 PM
To: 'Andy Walls'
Cc: ivtv-users@ivtvdriver.org
Subject: Re: [ivtv-users] Help With Troubleshooting
Andy,
Thank you for all the help. I will use whatever resources I can in order to
isolate and resolve this issue. To further the research a little, I decided
to try the Mythdora distro, and found some very interesting results.
Loading up and trying version 12.23 got me a 100% working system right away,
without any modifications. I then performed updates, and it all went to
hell. Didn't get the jumping distorted pictures, but it would lock up at a
blank screen anytime I tried live TV. I am going to go back and check the
version of the backend / frontends, where it was working and not, but then
this could be an entirely new issue, and not even related to the issue I am
hunting down.
I will post the information here when I get a chance. Maybe it will help...
-----Original Message-----
From: Andy Walls [mailto:awalls@md.metrocast.net]
Sent: Saturday, February 11, 2012 11:02 AM
To: Jeffrey Preston
Cc: ivtv-users@ivtvdriver.org
Subject: RE: [ivtv-users] Help With Troubleshooting
On Thu, 2012-02-09 at 11:22 -0600, Jeffrey Preston wrote:
> Andy,
>
> I have tried the following things:
>
> 1 - I disabled Hyperthreading on my P4 3.2Ghz CPU, and for several
> minutes, this seemed to resolve the issue, but then it started
> happening again, and the picture went distorted, with lots of
> skipping, and jumping. MythTV would sometimes exit with "Error
> opening jump program file", or "Video frame buffering failed too many
> times".
>
> 2 - I changed grub and rebooted the machine without the GUI loaded.
> So this accomplished two things....no nVidia driver loaded and no
> Window managers running, so a fairly clean system as far as processes
> running. I started mythbackend, but then it exited at the end (maybe
> it is supposed to do that). I then started a remote frontend and the
> first station it tuned to gave me the same trouble. Re-started the
> system with graphics again.
>
> 3 - I ran the command 'v4l2-ctl --log-status' several times, but could
> see nothing in the output that would give me any idea of the signal
> quality or strength. Maybe I am not looking at the output correctly.
You just want to see that a signal is present consistently. I suspect it
was.
> 4 - I issued the command "modprobe ivtv debug=0x4f", but could not
> find any logs in the folder you specified. I did look at the output
> from "dmesg" and found what I believe to be the results of the debug
> output from the ivtv module. However, there is only so much space, it
> seems, in the output from dmesg, and I could not find anything that
> was there that would give me a clue as to what is going on. Is there
> a way to get the debug mode for the ivtv module to redirect its output
> to a log file, so I can comb through it without worrying about the
> buffer space in dmesg?
That's a syslog daemon configuration. The ivtv driver uses KERN_INFO and
KERN_DEBUG (and KERN_ERROR and KERN_WARN) logging facilities.
As root, in /etc/rsyslod.conf or /etc/syslogd.conf you would want to direct
"kern.*" logging to /var/log/kernelmsgs or whatever. Then send syslod or
rsyslogd a SIGHUP to re-read the configuration:
# kill -HUP (process ID of system log daemon)
> 5 - I looked to see if I could load IRQbalance as you suggested, and
> found that it was already running under Mythbuntu 11.04. I then
> unloaded it, rebooted and tried again - no difference.
OK.
> I tried ftrace, but I believe it is beyond my abilities to configure
> and run at the moment. I do not know how to mount the debug file
> system or enable the option on the Ubuntu kernel.
I agree, it is complex. I really don't expect normal users to be able to
use it as an effective troubleshooting tool.
> I have seen the MythTV bug reports, but they do not seem to apply
> exactly to my situation, however, I cannot help but think that they
> are at least linked to what is going on with my system. I do not
> believe it is a latency issue, as adjusting the latency of the cards
> and other things on my system made no difference in the behavior of
> the analog tuner (PVR-150's) at all.
The PCI latency timer is just so that no one card can hog and hang the PCI
bus (remember the days of ISA cards?).
The PCI bus has a Round-Robin arbiter for bus access, and the latency timer
is the maximu amount of time that device can master the bus before having to
yield the bus back to the next bus grantee by the arbiter.
You can only set PCI latency timer values in multiples of 8 BTW. The PCI
clock runs at 33 MHz which makes 8 cycles to be approximately 250
nanoseconds (242 ns actually).
The ivtv driver tries to set the card's latency timer to allow its internal
DMA engine to have the PCI bus for 64 cycles (~2 microseconds) before the
card has to yield the PCI bus to another waiting PCI device.
You might want to check the latency timers of any PCI bridges upstream of
the PVR-150. If their latency timer is shorter than the PVR-150s, then the
latency timer of the PVR-150 doesn't really matter.
> I also do not think it has anything to do with changing channels, as
> I have had distorted video come up when first bring up my
> frontend...any frontend.
> I know I should not let this get to me, but I had a working system
> under Mythbuntu 8.10, and it doesn't work under 11.04. I found that
> the issue actually began, or has something to do with the changes
> between Mythbuntu 8.10 and 9.10. I don't know where the changes are
> located that would affect analog tuners, but I will guarantee you the
> source of the issue is there.
To find the root cause blindly but deterministically, with no guess-work,
one would have to
- be able to reproduce your problem with a known bad kernel
- have a known working kernel version
- do a git bisect on the kernel source between those two versions,
- rebuild the kernel and install it and test it
- iterate the bisection process until the bad kernel change is isolated
Each iteration takes ~30 minutes for me. Unfortunately that's time I really
don't have available. :(
> The problem still existed in Mythbuntu 10.10, as well.
>
> Perhaps someone that is familiar with the design architectures of
> Linux and Mythbuntu could point us all in the right direction.
>
> Andy, I want to thank you for your help so far. I would not have
> gotten this far without your help. Thanks for taking pity on an aging
> engineer.
You're welcome. Sorry I don't have a lot of time available lately.
Aging engineers have to watch out for each other. ;)
Regards,
Andy
> -----Original Message-----
> From: Andy Walls [mailto:awalls@md.metrocast.net]
> Sent: Saturday, January 28, 2012 8:25 AM
> To: jpreston@longlines.com
> Cc: ivtv-users@ivtvdriver.org
> Subject: Re: [ivtv-users] Help With Troubleshooting
>
> On Thu, 2012-01-26 at 18:39 -0600, jpreston@longlines.com wrote:
> > ==============Original message text=============== On Wed, 25 Jan
> > 2012
> > 9:40:11 pm CST Andy Walls wrote:
> > jpreston@longlines.com wrote:
> > >>Hello,
> > >>I have two servers running Mythbuntu 11.04 and MythTV 0.24.1.
> > >>I have
> > >>three PVR-150 tuners in the server configured as a slave backend,
> > >>and some digital tuners in the master backend. The problem I have
> > >>is that whenever the analog tuners are called for by any frontend
> > >>(even a local one), I get distorted video, with stuttering and
> > >>image freezes. There is currently a bug report in with MythTV on
> > >>a Analog Channel Change issue.
> > >>I have tried the work-a-round (tune a digital channel before using
> > >>one of the analog tuners), but it does not work for me. I finally
> > >>saw some troubleshooting steps for IVTV drivers, and followed some
> > >>of those steps.
> > >>The one that caught my attention, is the cat /dev/video1 >
> > >>test.mpg test. I looked at the file that was produced while the
> > >>command was active, and VLC would only play a few seconds of the
> > >>file and then skip to the end, and then terminate the viewing
> > >>window. The video jumped and was distorted, just the same as it
> > >>was in MythTV. So the issue, evidently, is not MythTV, but
> > >>perhaps with IVTV or some related item.
> > >>On a related note, the same hardware works without issues, when
> > >>running MythTV 0.21 under Mythbuntu 8.10.
> > >>What would you do next and where would you look for a>>resolution?
> > >>Thanks,
> > >>Jeff
>
> > >What are the kernel versions shown with uname -a?
>
> > Linux MYTHBE2 2.6.38-13-generic #54-Ubuntu SMP Tue Jan 3 13:44:52
> > UTC
> > 2012 i686 i686 i386 GNU/Linux
>
> OK, fairly modern; that's good. Do you happen to know the kernel
> version under Mythbuntu 8.10? (so I check the differences between the
> two kernels)
>
> > >What are the ivtv-driver versions shown with v4l2-ctl -d
> > >/dev/video1 --log-status?
>
> > ivtv1: Version: 1.4.2 Card: Hauppauge WinTV PVR-150
>
> OK, good.
>
> > >What other devices are shown sharing the interrupt lines with ivtv
> > >using cat /proc/interrupts?
>
> > CPU0 CPU1
> > 0: 580 0 IO-APIC-edge timer
> > 1: 3649 0 IO-APIC-edge i8042
> > 6: 3 0 IO-APIC-edge floppy
> > 7: 0 0 IO-APIC-edge parport0
> > 8: 1 0 IO-APIC-edge rtc0
> > 9: 0 0 IO-APIC-fasteoi acpi
> > 14: 63335 0 IO-APIC-edge ata_piix
> > 15: 1294927 0 IO-APIC-edge ata_piix
> > 16: 7153671 2827260 IO-APIC-fasteoi uhci_hcd:usb2,
uhci_hcd:usb5, nvidia
> > 17: 12238 7179 IO-APIC-fasteoi ivtv1
> > 18: 5690498 0 IO-APIC-fasteoi ata_piix,
uhci_hcd:usb4,eth0, Ensoniq AudioPCI
> > 19: 74042 0 IO-APIC-fasteoi uhci_hcd:usb3, ivtv2
> > 22: 1829 0 IO-APIC-fasteoi ivtv0
> > 23: 3 0 IO-APIC-fasteoi ehci_hcd:usb1
> > NMI: 0 0 Non-maskable interrupts
> > LOC: 14667959 15620982 Local timer interrupts
> > SPU: 0 0 Spurious interrupts
> > PMI: 0 0 Performance monitoring interrupts
> > IWI: 0 0 IRQ work interrupts
> > RES: 2407721 2344290 Rescheduling interrupts
> > CAL: 789 2664 Function call interrupts
> > TLB: 13663 20105 TLB shootdowns
> > TRM: 0 0 Thermal event interrupts
> > THR: 0 0 Threshold APIC interrupts
> > MCE: 0 0 Machine check exceptions
> > MCP: 540 540 Machine check polls
> > ERR: 0
> > MIS: 0
>
> This looks pretty imbalanced: CPU0 handles a lot more of the interrupt
service than CPU1. This imbalance of CPU0 handling most of the hardware
interrupts might be a source of problems, *if* CPU0 is actually busy with
higher priority interrupts (ata_piix, nvidia) when the ivtv* card interrupts
come in.
>
> The disk drive (ata_piix) and network card (eth0) interrupts have only
been handled by CPU0.
>
> Is "irqbalance" running on your system?
>
> For a back end system, I have to wonder what the nvidia card is doing for
you, assuming it it the only device generating interrupts on IRQ16.
> It has genearted 9.98M interrupts in the same time of 30.3M local timer
interrupts (1 nvidia card interrupt per every 3 timer interrupts).
>
>
>
> > Looks like one of the USB ports shares its IRQ with one of the
> > cards, but my issue happens with all three cards, so that doesn't
> > seem to be the likely cause of the issue.
>
> Right, especially if there are no USB devices connected to the port
> that
> IRQ19 serves.
>
> > >What is the cpu loading shown with top during video capture? Are
> > >there high numbers for user or io wait percentages?
> > Tasks: 146 total, 1 running, 145 sleeping, 0 stopped, 0 zombie
> > Cpu(s): 7.2%us, 4.0%sy, 0.0%ni, 88.8%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
> > Mem: 2059632k total, 1143540k used, 916092k free, 112388k buffers
> > Swap: 2095100k total, 0k used, 2095100k free, 688688k cached
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 2799 jpreston 20 0 214m 49m 24m S 2 2.5 36:37.44 knotify4
> > 7959 jpreston 20 0 72220 12m 9.8m S 2 0.6 0:04.54
xfce4-terminal
> > 1263 root 20 0 88408 51m 14m S 1 2.6 1:35.49 Xorg
> > 8028 jpreston 20 0 3360 504 440 S 1 0.0 0:00.35 cat
> > 8082 jpreston 20 0 2632 1136 848 R 1 0.1 0:00.08 top
> > 1345 jpreston 20 0 6464 2516 1788 S 0 0.1 0:06.88
xscreensaver
> > 1 root 20 0 3028 1792 1216 S 0 0.1 0:00.90 init
> > 2 root 20 0 0 0 0 S 0 0.0 0:00.04 kthreadd
> > 3 root 20 0 0 0 0 S 0 0.0 0:01.95 ksoftirqd/0
> > 5 root 20 0 0 0 0 S 0 0.0 0:03.60 kworker/u:0
> > 6 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
> > 7 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
> > 9 root 20 0 0 0 0 S 0 0.0 0:00.71 ksoftirqd/1
> > 11 root 0 -20 0 0 0 S 0 0.0 0:00.00 cpuset
> > 12 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
> > 13 root 0 -20 0 0 0 S 0 0.0 0:00.00 netns
> > 15 root 20 0 0 0 0 S 0 0.0 0:00.27 sync_supers
>
> > This was captured during a cat /dev/video1 > test.mpg, and the CPU
> > usage never went above a few percent for any process. The file was
> > still distorted and flawed.
>
> OK. Plenty of memory (894 MB RAM still free), plenty of CPU (88.8% idle),
nothing waiting on I/O (0.0% wa).
>
> 'cat' consuming only 1% of CPU may be a little low, since the ivtv driver
and cat collectively do some buffer copies in the read() call that cat makes
into the ivtv driver. (May indicate not many filled video buffers being
provided by the ivtv driver).
>
>
> Given this, the best guesses I have are:
> 1. an interrupt handling latency problem.
> 2. a PCI bus/chipset problem: DMA transfers aborting or IO
> transactions to registers and memory on the CX23416 chip are being
> aborted 3. an ivtv driver bug/race in setting up the CX23416 decoder,
> that causes problems in the decoder actually running 4. a cx25840
> driver bug in setting up the CX25843 chip to lock on to the video
> signal and digitize it properly 5. an analog tuner driver bug that
> causes tuning to fail
>
> > Where else can I look for this? Do you have any other ideas?
>
> For #4 and #5 above, the output of several runs of 'v4l2-ctl --log-status'
should let you know if the CX25843 has a good video signal.
>
> For #2 and #3, setting the debug=... option to the ivtv driver in
/etc/modprobe.conf or on the modprobe ivtv command line will turn on various
levels of logging in the ivtv driver (with messages emitted to
/var/log/messages normally).
>
> $ modinfo ivtv
> [...]
> parm: debug:Debug level (bitmask). Default: 0
> 1/0x0001: warning
> 2/0x0002: info
> 4/0x0004: mailbox
> 8/0x0008: ioctl
> 16/0x0010: file
> 32/0x0020: dma
> 64/0x0040: irq
> 128/0x0080: decoder
> 256/0x0100: yuv
> 512/0x0200: i2c
> 1024/0x0400: high volume
> [...]
>
> So maybe, as root,
>
> # modprobe -r ivtv
> # modprobe ivtv debug=0x4f (or =0x44f)
>
> And then do a 'cat /dev/videoN > foo.mpg' and examine all the debug spew
in /var/log/messages to see what is wrong.
>
>
> For #1, I'd experiment with how does video capture behave with
>
> a. having irqbalance running (or not running) b. booting into only
> run-level 3 (text console, no GUI/X) and unloading the nvidia driver
> module
>
>
> I would also look at IRQ handling latency with ftrace and other kernel
tracing mechanisms:
>
> http://lwn.net/Articles/322666/
> http://lwn.net/Articles/322731/
> http://lwn.net/Articles/366796/
> http://lwn.net/Articles/365835/
> http://www.spinics.net/lists/linux-media/msg15762.html
>
> Regards,
> Andy
>
> > Thank you much for your assistance so far.
>
> > Jeff
> >
>
>
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.1901 / Virus Database: 2109/4772 - Release Date:
> 01/28/12
>
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2112/4804 - Release Date: 02/11/12
_______________________________________________
ivtv-users mailing list
ivtv-users@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-users -----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2112/4807 - Release Date: 02/13/12
_______________________________________________
ivtv-users mailing list
ivtv-users@ivtvdriver.org
http://ivtvdriver.org/mailman/listinfo/ivtv-users