Mailing List Archive: Traffic issues Dom0

Traffic issues Dom0 > DomU

Jan 10, 2011, 11:19 PM

Post #1 of 12 (3851 views)

Hi there,

We have a very strange issues on our new Xen platform, i cant seem to get
everything up and running without performance issues. So far all our other
xen servers work perfectly but these new ones act strange.

Server Configuration:
HP Proliant DL360 G2
2x Quad-core Xeon
32 GB 667Mhz Memory
2x 72GB SAS 10krpm disks
2x 300GB SAS 10rpm disks
2x Gbit Nic Onboard

cu1:~# uname -a
Linux cu1.wipa.nl 2.6.26-2-xen-amd64 #1 SMP Thu Nov 25 06:39:26 UTC 2010
x86_64 GNU/Linux
cu1:~#

eth0 Link encap:Ethernet HWaddr 00:26:55:7b:9a:88
inet addr:83.96.235.37 Bcast:83.96.235.127 Mask:255.255.255.128
inet6 addr: fe80::226:55ff:fe7b:9a88/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:558266 errors:0 dropped:0 overruns:0 frame:0
TX packets:47770 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:29222362 (27.8 MiB) TX bytes:5541863 (5.2 MiB)

eth1 Link encap:Ethernet HWaddr 00:26:55:7b:aa:ec
inet addr:172.50.20.150 Bcast:172.50.20.255 Mask:255.255.255.0
inet6 addr: fe80::226:55ff:fe7b:aaec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1024 (1024.0 B) TX bytes:492 (492.0 B)
Interrupt:19 Memory:fa000000-fa012100

peth0 Link encap:Ethernet HWaddr 00:26:55:7b:9a:88
inet6 addr: fe80::226:55ff:fe7b:9a88/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:1174819 errors:0 dropped:0 overruns:0 frame:0
TX packets:403716 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:360329375 (343.6 MiB) TX bytes:53501555 (51.0 MiB)
Interrupt:18 Memory:f8000000-f8012100

vif12.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:33426 errors:0 dropped:0 overruns:0 frame:0
TX packets:459074 errors:0 dropped:225 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:1907670 (1.8 MiB) TX bytes:111555928 (106.3 MiB)

vif6.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:161967 errors:0 dropped:0 overruns:0 frame:0
TX packets:647936 errors:0 dropped:207 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:18441838 (17.5 MiB) TX bytes:149516297 (142.5 MiB)

vif7.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:81477 errors:0 dropped:0 overruns:0 frame:0
TX packets:514458 errors:0 dropped:155 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:8176067 (7.7 MiB) TX bytes:83209185 (79.3 MiB)

Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- ns1.itrends.eu anywhere PHYSDEV match
--physdev-in vif6.0
ACCEPT udp -- anywhere anywhere PHYSDEV match
--physdev-in vif6.0 udp spt:bootpc dpt:bootps
ACCEPT all -- ip-space.colo.itrends.eu anywhere PHYSDEV
match --physdev-in vif7.0
ACCEPT udp -- anywhere anywhere PHYSDEV match
--physdev-in vif7.0 udp spt:bootpc dpt:bootps
ACCEPT all -- web01.touch.itrends.eu anywhere PHYSDEV
match --physdev-in vif12.0
ACCEPT udp -- anywhere anywhere PHYSDEV match
--physdev-in vif12.0 udp spt:bootpc dpt:bootps

For now not all Iptables rules are here, i flushed it and rebooted the DomU.
But the ones i rebooted you can see up here in the iptables, it sets the
forwards.

cu1:/proc/sys/net/ipv4/conf/all# cat proxy_arp
1
cu1:/proc/sys/net/ipv4/conf/all#

cu1:/proc/sys/net/ipv4/conf/all# cat forwarding
1
cu1:/proc/sys/net/ipv4/conf/all#

Example host file:

#
# Kernel + memory size
#
kernel = '/boot/vmlinuz-2.6.26-2-xen-amd64'
ramdisk = '/boot/initrd.img-2.6.26-2-xen-amd64'
memory = '512'
extra = "console=tty xencons=tty"
#
# Disk device(s).
#
root = '/dev/sda2 ro'
disk = [
'phy:/dev/vg/ns1.xxxxx.eu-swap,sda1,w',
'phy:/dev/vg/ns1.xxxxx.eu-disk,sda2,w',
]

#
# Hostname
#
name = 'ns1. xxxxx.eu'

#
# Networking
#
vif = [ 'ip=XX.XX.XX.XX.9,mac=00:16:3E:F5:16:21' ]

#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

--------
So far all well, it looks like everything has been setup correctly.

When i ping from the dom0 to the domU:

--- XX.XX.XX.9 ping statistics ---
19 packets transmitted, 12 received, 36% packet loss, time 18021ms
rtt min/avg/max/mdev = 0.032/0.181/1.794/0.486 ms

This means that it drops a lot of connections and this domU seriously can't
function like this. I've never seen this on any of our other Xen servers so
this one is really bugging me.

Anybody else had this problem before? The log is not showing anything (no
errors or faults)

------------------------------------------------------------------------

Met vriendelijk groet, / Kind regards,
WIPA Nederland B.V.

Wieger Bontekoe
email: w.bontekoe@wipabv.nl

Telefoon: 036 - 522 3633 (Zeewolde)
Telefoon: 070 - 711 0260 (Den Haag)
------------------------------------------------------------------------

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Jan 27, 2011, 12:39 PM

Post #2 of 12 (3736 views)

Permalink

> When i ping from the dom0 to the domU:
>
> --- XX.XX.XX.9 ping statistics ---
> 19 packets transmitted, 12 received, 36% packet loss, time 18021ms
> rtt min/avg/max/mdev = 0.032/0.181/1.794/0.486 ms
>
> This means that it drops a lot of connections and this domU seriously can't
> function like this. I've never seen this on any of our other Xen servers so
> this one is really bugging me.
>
> Anybody else had this problem before? The log is not showing anything (no
> errors or faults)

You tried the usual assortment of turning gso/tso/whatnot on/off? Look
in the xen-devel mailing list for the proper commands.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

w.bontekoe at wipabv

Jan 27, 2011, 12:43 PM

Post #3 of 12 (3731 views)

Permalink

Yes i did, exactly. But also there i found no changes. Every VM that i move to this host has the exact same problem though they use seeral kernals/versions. I compared all VM's, nothing.

I even did a reinstall of the dom0 twice, updated all device drivers (HP PSP) .. Nothing.

Its really a problem on the Xen bridge (peth0), even in NAT mode....

----------------------
Met vriendelijk groet / Kind regards,
WIPA Nederland B.V. / Information Trends B.V.

Wieger Bontekoe
Tel.: 06 - 24378263

Verzonden vanaf mijn iPhone 4 16GB
----------------------
Op al onze producten en diensten zijn onze algemene voorwaarden van toepassing.
----------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named.

If you are not the named addressee you should not disseminate, distribute or
copy this e-mail. Please notify the sender immediately by e-mail if you have
received this e-mail by mistake and delete this e-mail from your system.

If you are not the intended recipient you are notified that disclosing,
copying, distributing or taking any action in reliance on the contents of this
information is strictly prohibited.

Op 27 jan. 2011 om 21:40 heeft "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> het volgende geschreven:

>> When i ping from the dom0 to the domU:
>>
>> --- XX.XX.XX.9 ping statistics ---
>> 19 packets transmitted, 12 received, 36% packet loss, time 18021ms
>> rtt min/avg/max/mdev = 0.032/0.181/1.794/0.486 ms
>>
>> This means that it drops a lot of connections and this domU seriously can't
>> function like this. I've never seen this on any of our other Xen servers so
>> this one is really bugging me.
>>
>> Anybody else had this problem before? The log is not showing anything (no
>> errors or faults)
>
> You tried the usual assortment of turning gso/tso/whatnot on/off? Look
> in the xen-devel mailing list for the proper commands.
>
>

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Feb 2, 2011, 12:06 PM

Post #4 of 12 (3719 views)

Permalink

On Thu, Jan 27, 2011 at 09:43:25PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Yes i did, exactly. But also there i found no changes. Every VM that i move to this host has the exact same problem though they use seeral kernals/versions. I compared all VM's, nothing.
>
> I even did a reinstall of the dom0 twice, updated all device drivers (HP PSP) .. Nothing.

What happens if you modify the size of the ping? Say ping -s80 or ping -s2000 does it work better or worst?

>
> Its really a problem on the Xen bridge (peth0), even in NAT mode....
>
>
> ----------------------
> Met vriendelijk groet / Kind regards,
> WIPA Nederland B.V. / Information Trends B.V.
>
> Wieger Bontekoe
> Tel.: 06 - 24378263
>
> Verzonden vanaf mijn iPhone 4 16GB
> ----------------------
> Op al onze producten en diensten zijn onze algemene voorwaarden van toepassing.
> ----------------------
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system manager.
> This message contains confidential information and is intended only for the
> individual named.
>
> If you are not the named addressee you should not disseminate, distribute or
> copy this e-mail. Please notify the sender immediately by e-mail if you have
> received this e-mail by mistake and delete this e-mail from your system.
>
> If you are not the intended recipient you are notified that disclosing,
> copying, distributing or taking any action in reliance on the contents of this
> information is strictly prohibited.
>
> Op 27 jan. 2011 om 21:40 heeft "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> het volgende geschreven:
>
> >> When i ping from the dom0 to the domU:
> >>
> >> --- XX.XX.XX.9 ping statistics ---
> >> 19 packets transmitted, 12 received, 36% packet loss, time 18021ms
> >> rtt min/avg/max/mdev = 0.032/0.181/1.794/0.486 ms
> >>
> >> This means that it drops a lot of connections and this domU seriously can't
> >> function like this. I've never seen this on any of our other Xen servers so
> >> this one is really bugging me.
> >>
> >> Anybody else had this problem before? The log is not showing anything (no
> >> errors or faults)
> >
> > You tried the usual assortment of turning gso/tso/whatnot on/off? Look
> > in the xen-devel mailing list for the proper commands.
> >
> >
>

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

w.bontekoe at wipabv

Feb 2, 2011, 12:07 PM

Post #5 of 12 (3819 views)

Permalink

As far as i can see now, makes no difference.

When i create a new VM from the dom0 everything is fine, when i move a
DomU from another Dom0 with the same kernel i get the ping loss.

----------------------
Met vriendelijk groet / Kind regards,
WIPA Nederland B.V. / Information Trends B.V.

Wieger Bontekoe
Tel.: 06 - 24378263

Verzonden vanaf mijn iPhone 4 16GB
----------------------
Op al onze producten en diensten zijn onze algemene voorwaarden van toepassing.
----------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named.

If you are not the named addressee you should not disseminate, distribute or
copy this e-mail. Please notify the sender immediately by e-mail if you have
received this e-mail by mistake and delete this e-mail from your system.

If you are not the intended recipient you are notified that disclosing,
copying, distributing or taking any action in reliance on the contents of this
information is strictly prohibited.

Op 2 feb. 2011 om 21:06 heeft "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> het volgende geschreven:

> On Thu, Jan 27, 2011 at 09:43:25PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
>> Yes i did, exactly. But also there i found no changes. Every VM that i move to this host has the exact same problem though they use seeral kernals/versions. I compared all VM's, nothing.
>>
>> I even did a reinstall of the dom0 twice, updated all device drivers (HP PSP) .. Nothing.
>
> What happens if you modify the size of the ping? Say ping -s80 or ping -s2000 does it work better or worst?
>
>>
>> Its really a problem on the Xen bridge (peth0), even in NAT mode....
>>
>>
>> ----------------------
>> Met vriendelijk groet / Kind regards,
>> WIPA Nederland B.V. / Information Trends B.V.
>>
>> Wieger Bontekoe
>> Tel.: 06 - 24378263
>>
>> Verzonden vanaf mijn iPhone 4 16GB
>> ----------------------
>> Op al onze producten en diensten zijn onze algemene voorwaarden van toepassing.
>> ----------------------
>> This email and any files transmitted with it are confidential and intended
>> solely for the use of the individual or entity to whom they are addressed.
>> If you have received this email in error please notify the system manager.
>> This message contains confidential information and is intended only for the
>> individual named.
>>
>> If you are not the named addressee you should not disseminate, distribute or
>> copy this e-mail. Please notify the sender immediately by e-mail if you have
>> received this e-mail by mistake and delete this e-mail from your system.
>>
>> If you are not the intended recipient you are notified that disclosing,
>> copying, distributing or taking any action in reliance on the contents of this
>> information is strictly prohibited.
>>
>> Op 27 jan. 2011 om 21:40 heeft "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> het volgende geschreven:
>>
>>>> When i ping from the dom0 to the domU:
>>>>
>>>> --- XX.XX.XX.9 ping statistics ---
>>>> 19 packets transmitted, 12 received, 36% packet loss, time 18021ms
>>>> rtt min/avg/max/mdev = 0.032/0.181/1.794/0.486 ms
>>>>
>>>> This means that it drops a lot of connections and this domU seriously can't
>>>> function like this. I've never seen this on any of our other Xen servers so
>>>> this one is really bugging me.
>>>>
>>>> Anybody else had this problem before? The log is not showing anything (no
>>>> errors or faults)
>>>
>>> You tried the usual assortment of turning gso/tso/whatnot on/off? Look
>>> in the xen-devel mailing list for the proper commands.
>>>
>>>
>>
>
>

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Feb 2, 2011, 12:34 PM

Post #6 of 12 (3718 views)

Permalink

On Wed, Feb 02, 2011 at 09:07:40PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> As far as i can see now, makes no difference.
>
> When i create a new VM from the dom0 everything is fine, when i move a
> DomU from another Dom0 with the same kernel i get the ping loss.

And if you move it back?

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

w.bontekoe at wipabv

Feb 2, 2011, 12:35 PM

Post #7 of 12 (3731 views)

Permalink

Then the problems are gone

----------------------
Met vriendelijk groet / Kind regards,
WIPA Nederland B.V. / Information Trends B.V.

Wieger Bontekoe
Tel.: 06 - 24378263

Verzonden vanaf mijn iPhone 4 16GB
----------------------
Op al onze producten en diensten zijn onze algemene voorwaarden van toepassing.
----------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named.

If you are not the named addressee you should not disseminate, distribute or
copy this e-mail. Please notify the sender immediately by e-mail if you have
received this e-mail by mistake and delete this e-mail from your system.

If you are not the intended recipient you are notified that disclosing,
copying, distributing or taking any action in reliance on the contents of this
information is strictly prohibited.

Op 2 feb. 2011 om 21:35 heeft "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com> het volgende geschreven:

> On Wed, Feb 02, 2011 at 09:07:40PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
>> As far as i can see now, makes no difference.
>>
>> When i create a new VM from the dom0 everything is fine, when i move a
>> DomU from another Dom0 with the same kernel i get the ping loss.
>
> And if you move it back?
>
>

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Feb 17, 2011, 6:32 AM

Post #8 of 12 (3742 views)

Permalink

On Wed, Feb 02, 2011 at 09:35:13PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Then the problems are gone

Have you thought about switching the Ethernet cables or the network
port switches to see if anything changes?

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

RE: Traffic issues Dom0 > DomU [ In reply to ]

w.bontekoe at wipabv

Feb 20, 2011, 5:00 AM

Post #9 of 12 (3686 views)

Permalink

Yes, i bought new switches (HP Procurve 2810) - changed the NIC (new Intel) and used brand new cables. It really is an software issue.

------------------------------------------------------------------------

Met vriendelijk groet,
WIPA Nederland B.V.

Wieger Bontekoe
email: w.bontekoe@wipabv.nl

Telefoon: 036 - 522 3633 (Zeewolde)
Telefoon: 070 - 711 0260 (Den Haag)
------------------------------------------------------------------------
WIPA Nederland B.V. is onderdeel van Information Trends B.V. en Fascom Beheer
B.V. en geregistreerd bij het handelsregister Gooi-, Eem-, en Flevoland onder
nummer 32138703. Bankrekening 55.80.33.431 t.n.v. WIPA Nederland B.V. te
Zeewolde.
________________________________________
Van: Konrad Rzeszutek Wilk [konrad.wilk@oracle.com]
Verzonden: donderdag 17 februari 2011 15:32
Aan: W. Bontekoe - WIPA Nederland B.V.
CC: xen-bugs@lists.xensource.com
Onderwerp: Re: [Xen-bugs] Traffic issues Dom0 > DomU

On Wed, Feb 02, 2011 at 09:35:13PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Then the problems are gone

Have you thought about switching the Ethernet cables or the network
port switches to see if anything changes?

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Mar 9, 2011, 10:08 AM

Post #10 of 12 (3678 views)

Permalink

On Sun, Feb 20, 2011 at 02:00:53PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Yes, i bought new switches (HP Procurve 2810) - changed the NIC (new Intel) and used brand new cables. It really is an software issue.

Let me double-check this email thread, but you are sure that the machines
are exactly identical? As in even in the firmware and memory amount?

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

RE: Traffic issues Dom0 > DomU [ In reply to ]

w.bontekoe at wipabv

Mar 13, 2011, 4:29 AM

Post #11 of 12 (3676 views)

Permalink

Yes, i confirmed this.

Both machines were patched fully at the exact same date and time and have the same hardware/software configuration. There was absolutely no difference between both machines. I just reinstalled 2 new ones (same HP Procurve DL 360 G5 models) and for now i'm experiencing the exact same problem with several VM's

________________________________________
Van: Konrad Rzeszutek Wilk [konrad.wilk@oracle.com]
Verzonden: woensdag 9 maart 2011 18:08
Aan: W. Bontekoe - WIPA Nederland B.V.
CC: xen-bugs@lists.xensource.com
Onderwerp: Re: [Xen-bugs] Traffic issues Dom0 > DomU

On Sun, Feb 20, 2011 at 02:00:53PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Yes, i bought new switches (HP Procurve 2810) - changed the NIC (new Intel) and used brand new cables. It really is an software issue.

Let me double-check this email thread, but you are sure that the machines
are exactly identical? As in even in the firmware and memory amount?

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs

Re: Traffic issues Dom0 > DomU [ In reply to ]

konrad.wilk at oracle

Mar 16, 2011, 11:43 AM

Post #12 of 12 (3622 views)

Permalink

On Sun, Mar 13, 2011 at 12:29:53PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> Yes, i confirmed this.
>
> Both machines were patched fully at the exact same date and time and have the same hardware/software configuration. There was absolutely no difference between both machines. I just reinstalled 2 new ones (same HP Procurve DL 360 G5 models) and for now i'm experiencing the exact same problem with several VM's

And you tried the ethtool tool to turn of tso, gso in the DomU?

You should open a bug about this.
>
> ________________________________________
> Van: Konrad Rzeszutek Wilk [konrad.wilk@oracle.com]
> Verzonden: woensdag 9 maart 2011 18:08
> Aan: W. Bontekoe - WIPA Nederland B.V.
> CC: xen-bugs@lists.xensource.com
> Onderwerp: Re: [Xen-bugs] Traffic issues Dom0 > DomU
>
> On Sun, Feb 20, 2011 at 02:00:53PM +0100, W. Bontekoe - WIPA Nederland B.V. wrote:
> > Yes, i bought new switches (HP Procurve 2810) - changed the NIC (new Intel) and used brand new cables. It really is an software issue.
>
> Let me double-check this email thread, but you are sure that the machines
> are exactly identical? As in even in the firmware and memory amount?
>
>
>
>
>
>
> _______________________________________________
> Xen-bugs mailing list
> Xen-bugs@lists.xensource.com
> http://lists.xensource.com/xen-bugs

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs