Hi all!
I'm using Xen to emulate huge networks (30 domains with 4 to 20
interfaces each), so I need lots of interfaces, which in turn need a lot
of (dynamic) interrupts (one per interface). But when I'm exceeding the
256 interrupt limit of the kernel, domain0 panics giving a me hint to
increase NR_IRQS. So I did what I was told and changed NR_IRQS from 256
to 512: Since in
/usr/src/linux/include/asm-xen/asm-i386/mach-xen/irq_vectors.h line 111,
NR_IRQS is defined as
#define NR_IRQS (NR_PIRQS + NR_DYNIRQS)
I've instead increased NR_DYNIRQS from 128 to 384 to change NR_IRQS from
256 to 512. The xen0 does not panic anymore, fine, but things still do
not work.
Now. when booting a xenU domain when all interrupts of xen0 below 256
are already in use, the network frontend complaints about beeing unable
to connect the virtual interfaces. These are the last lines during the
boot of xenU:
[...]
Xen virtual console successfully installed as tty
Event-channel device installed.
xen_blk: Initialising virtual block device driver
xen_net: Initialising virtual ethernet driver.
xen_net: Failed to connect all virtual interfaces: err=-100
NET: Registered protocol family 2
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 4096 bind 8192)
NET: Registered protocol family 8
NET: Registered protocol family 20
That's all. Aftes this, the xenU domain hangs. Taking a look at the
allocated interrupts of domain 0, it seems as if the block backend makes
use of the new interrupts while the network backend doesn't:
root@tb1:~ >cat /proc/interrupts |tail -33
235: 1776 Dynamic-irq vif76.18
236: 2149 Dynamic-irq vif76.19
237: 11213 Dynamic-irq blkif-backend
238: 3 Dynamic-irq vif77.0
239: 30 Dynamic-irq vif77.1
240: 1788 Dynamic-irq vif77.2
241: 2145 Dynamic-irq vif77.3
242: 13147 Dynamic-irq vif77.4
243: 708 Dynamic-irq vif77.5
244: 303 Dynamic-irq vif77.6
245: 1793 Dynamic-irq vif77.7
246: 2128 Dynamic-irq vif77.8
247: 11376 Dynamic-irq vif77.9
248: 8 Dynamic-irq vif77.10
249: 30 Dynamic-irq vif77.11
250: 1813 Dynamic-irq vif77.12
251: 2132 Dynamic-irq vif77.13
252: 3916 Dynamic-irq vif77.14
253: 8 Dynamic-irq vif77.15
254: 165 Dynamic-irq vif77.16
255: 755 Dynamic-irq vif77.17
256: 0 Dynamic-irq vif77.18
257: 0 Dynamic-irq vif77.19
258: 0 Dynamic-irq blkif-backend
259: 0 Dynamic-irq blkif-backend
260: 0 Dynamic-irq blkif-backend
261: 0 Dynamic-irq blkif-backend
262: 0 Dynamic-irq blkif-backend
263: 0 Dynamic-irq blkif-backend
264: 0 Dynamic-irq blkif-backend
265: 0 Dynamic-irq blkif-backend
NMI: 0
ERR: 0
One can see that the last 8 domains can't allocate interrupts for their
network interfaces. They only allocate one interupts for the block
device backend of the root filesystem.
What have I done wrong? What do I have to do to get things working? I've
tried to get some hints from the (kernel) sources, but that's not that
easy, though...
Mark.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
I'm using Xen to emulate huge networks (30 domains with 4 to 20
interfaces each), so I need lots of interfaces, which in turn need a lot
of (dynamic) interrupts (one per interface). But when I'm exceeding the
256 interrupt limit of the kernel, domain0 panics giving a me hint to
increase NR_IRQS. So I did what I was told and changed NR_IRQS from 256
to 512: Since in
/usr/src/linux/include/asm-xen/asm-i386/mach-xen/irq_vectors.h line 111,
NR_IRQS is defined as
#define NR_IRQS (NR_PIRQS + NR_DYNIRQS)
I've instead increased NR_DYNIRQS from 128 to 384 to change NR_IRQS from
256 to 512. The xen0 does not panic anymore, fine, but things still do
not work.
Now. when booting a xenU domain when all interrupts of xen0 below 256
are already in use, the network frontend complaints about beeing unable
to connect the virtual interfaces. These are the last lines during the
boot of xenU:
[...]
Xen virtual console successfully installed as tty
Event-channel device installed.
xen_blk: Initialising virtual block device driver
xen_net: Initialising virtual ethernet driver.
xen_net: Failed to connect all virtual interfaces: err=-100
NET: Registered protocol family 2
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 4096 bind 8192)
NET: Registered protocol family 8
NET: Registered protocol family 20
That's all. Aftes this, the xenU domain hangs. Taking a look at the
allocated interrupts of domain 0, it seems as if the block backend makes
use of the new interrupts while the network backend doesn't:
root@tb1:~ >cat /proc/interrupts |tail -33
235: 1776 Dynamic-irq vif76.18
236: 2149 Dynamic-irq vif76.19
237: 11213 Dynamic-irq blkif-backend
238: 3 Dynamic-irq vif77.0
239: 30 Dynamic-irq vif77.1
240: 1788 Dynamic-irq vif77.2
241: 2145 Dynamic-irq vif77.3
242: 13147 Dynamic-irq vif77.4
243: 708 Dynamic-irq vif77.5
244: 303 Dynamic-irq vif77.6
245: 1793 Dynamic-irq vif77.7
246: 2128 Dynamic-irq vif77.8
247: 11376 Dynamic-irq vif77.9
248: 8 Dynamic-irq vif77.10
249: 30 Dynamic-irq vif77.11
250: 1813 Dynamic-irq vif77.12
251: 2132 Dynamic-irq vif77.13
252: 3916 Dynamic-irq vif77.14
253: 8 Dynamic-irq vif77.15
254: 165 Dynamic-irq vif77.16
255: 755 Dynamic-irq vif77.17
256: 0 Dynamic-irq vif77.18
257: 0 Dynamic-irq vif77.19
258: 0 Dynamic-irq blkif-backend
259: 0 Dynamic-irq blkif-backend
260: 0 Dynamic-irq blkif-backend
261: 0 Dynamic-irq blkif-backend
262: 0 Dynamic-irq blkif-backend
263: 0 Dynamic-irq blkif-backend
264: 0 Dynamic-irq blkif-backend
265: 0 Dynamic-irq blkif-backend
NMI: 0
ERR: 0
One can see that the last 8 domains can't allocate interrupts for their
network interfaces. They only allocate one interupts for the block
device backend of the root filesystem.
What have I done wrong? What do I have to do to get things working? I've
tried to get some hints from the (kernel) sources, but that's not that
easy, though...
Mark.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users