Mailing List Archive: Xen questions - Swap, networking and errors

Xen questions - Swap, networking and errors

Jul 6, 2005, 12:00 PM

Post #1 of 5 (629 views)

Hi,

This is my first time trying Xen. I have installed Xen-2.0.6, on a
Debian Sarge server. I have a few questions that I haven't been able to
find answers to in the documentation or FAQs.

1)When I boot xen0, the specified amount of RAM shows up in top
correctly, but the swap partitions I have are not activated. When I
attempt to activate swap myself, I recieve the following error message:
xen:/# swapon /dev/hde2
swapon: /dev/hde2: No such device or address
The server has a mirrored RAID1 setup, and both MD devices are detected
and mounted properly. However, I am confused about why the kernel does
not see the hard disk partitions themselves. The disks are SATA drives,
and so at first I thought perhaps it didn't recongize the drives, but if
that were the case, I don't think either MD device would be detected
properly. Is there a boot option that I have not seen to address this
issue?

As a side note, what is a proper amount of RAM needed for xen0? I have
not found any documentation really stating how much RAM is need for the
priveledged Xen instance. The server itself has 4GB of RAM. Should
xen0 have all 4GB of RAM to partition to the xenU domains?

2)With the default xend config, when I start xend the bridge device
xen-br0 is created. However, after that I lose all network activity to
xen0. I noticed that no vif devices were created, only xen-br0.
Initially, xen-br0 doesn't have an IP. I've tried setting it to the IP
of eth0, and a 192.168.0.x IP, but I still cannot get networking up.

I changed the xend config to use the network-route rather than network,
and when I start xend, networking doesn't go down, but xen-br0 is not
created. I am not sure why the default network config is not working.
Do I need to make changes to the routing tables?

3)When I create my test xenU instance, it appears to crash. Here is the
output from xend-debug.log:

VIRTUAL MEMORY ARRANGEMENT:
Loaded kernel: c0100000->c0341b84
Init. ramdisk: c0342000->c0342000
Phys-Mach map: c0342000->c0362000
Page tables: c0362000->c0364000
Start info: c0364000->c0365000
Boot stack: c0365000->c0366000
TOTAL: c0000000->c0400000
ENTRY ADDRESS: c0100000
ioctl: LOOP_SET_FD: Device or resource busy
vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
bridge=xen-br0
recv_fe_driver_status> {'status': 1}

recv_fe_driver_status>

recv_fe_interface_connect {'tx_shmem_frame': 6874, 'rx_shmem_frame':
6873, 'handle': 0}
VirqClient.virqReceived> 4
vif-route down vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
bridge=xen-br0

I am not sure why the error message LOOP_SET_FD: Device or resource busy
appears, since the loopback device I created the root filesystem on is
not mounted or being used.

I have attached my xend-config and xenU config that appears to be
crashing. Any help that can be offered is appreciated, especially links
to documentation that I might have missed.

Regards,

Dan Lang
--
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123

Re: Xen questions - Swap, networking and errors [ In reply to ]

maw48 at cam

Jul 6, 2005, 4:27 PM

Post #2 of 5 (616 views)

Permalink

> This is my first time trying Xen. I have installed Xen-2.0.6, on a
> Debian Sarge server. I have a few questions that I haven't been able to
> find answers to in the documentation or FAQs.

OK, welcome to the community! I'll try and answer your questions. If
there's anything you feel should be in a FAQ, feel free to get a Wiki
account and add it at http://wiki.xensource.com.

> 1)When I boot xen0, the specified amount of RAM shows up in top
> correctly, but the swap partitions I have are not activated. When I
> attempt to activate swap myself, I recieve the following error message:
> xen:/# swapon /dev/hde2
> swapon: /dev/hde2: No such device or address
> The server has a mirrored RAID1 setup, and both MD devices are detected
> and mounted properly. However, I am confused about why the kernel does
> not see the hard disk partitions themselves. The disks are SATA drives,
> and so at first I thought perhaps it didn't recongize the drives, but if
> that were the case, I don't think either MD device would be detected
> properly. Is there a boot option that I have not seen to address this

Are all the drives in the machine the same? Are other partitions on hde in
the RAID configuration.

If the drives are on different controllers (e.g. a couple of PATA and a
SATA, as on some systems) then the default kernel config may not be
building the drivers for hde but still builds enough drivers for the MD
devices to work.

> As a side note, what is a proper amount of RAM needed for xen0? I have
> not found any documentation really stating how much RAM is need for the
> priveledged Xen instance. The server itself has 4GB of RAM. Should
> xen0 have all 4GB of RAM to partition to the xenU domains?

RAM that you give to dom0 can't be allocated to domUs. 256MB should
probably be enough for a slim dom0. If you use LVM snapshots that'll need
to be larger though.

You can use the "balloon driver" to reduce the size of dom0 after booting
if you need more memory for other domains.

> 2)With the default xend config, when I start xend the bridge device
> xen-br0 is created. However, after that I lose all network activity to
> xen0. I noticed that no vif devices were created, only xen-br0.
> Initially, xen-br0 doesn't have an IP. I've tried setting it to the IP
> of eth0, and a 192.168.0.x IP, but I still cannot get networking up.

Sounds a bit broken... What should happen is that eth0's IP is transferred
to xen-br0.

Do you have any existing bridge setup? Anything else unusual in your
networking config?

> I changed the xend config to use the network-route rather than network,
> and when I start xend, networking doesn't go down, but xen-br0 is not
> created. I am not sure why the default network config is not working.
> Do I need to make changes to the routing tables?

You won't get a xen-br0 if you're using the routed config - you don't need
it. Don't know about the routing tables as I've never used the routed
config myself.

<snip>
> ioctl: LOOP_SET_FD: Device or resource busy
<snip>
> vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12

> I am not sure why the error message LOOP_SET_FD: Device or resource busy
> appears, since the loopback device I created the root filesystem on is
> not mounted or being used.

What does ls -l /dev/loop* give you?

Cheers,
Mark

> I have attached my xend-config and xenU config that appears to be
> crashing. Any help that can be offered is appreciated, especially links
> to documentation that I might have missed.
>
> Regards,
>
> Dan Lang
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: Spam:*, Xen questions - Swap, networking and errors [ In reply to ]

marcusbrutus at internode

Jul 6, 2005, 7:34 PM

Post #3 of 5 (613 views)

Permalink

Daniel Lang wrote:

>1)When I boot xen0, the specified amount of RAM shows up in top
>correctly, but the swap partitions I have are not activated. When I
>attempt to activate swap myself, I recieve the following error message:
> xen:/# swapon /dev/hde2
> swapon: /dev/hde2: No such device or address
>The server has a mirrored RAID1 setup, and both MD devices are detected
>and mounted properly. However, I am confused about why the kernel does
>not see the hard disk partitions themselves. The disks are SATA drives,
>and so at first I thought perhaps it didn't recongize the drives, but if
>that were the case, I don't think either MD device would be detected
>properly. Is there a boot option that I have not seen to address this
>issue?
>
Make sure the device exists with #fdisk -l /dev/hde
I guess it's possible that your drives are now allocated a different node.
On your original kernel, it would appear from what you're saying,
that your SATA drives all came up as hdX.
However, on you dom0 kernel, they are sdX.

>As a side note, what is a proper amount of RAM needed for xen0? I have
>not found any documentation really stating how much RAM is need for the
>priveledged Xen instance. The server itself has 4GB of RAM. Should
>xen0 have all 4GB of RAM to partition to the xenU domains?
>
>
The kernel stanza in grub's menu.lst specifies the amount of memory for
dom0 to use, not how much it has to share. I'm not certain about the actual
memory overheads involved in running domains, but I've found that 128MB
is more than enough to run a couple of domains, NFS server, etc.

>2)With the default xend config, when I start xend the bridge device
>xen-br0 is created. However, after that I lose all network activity to
>xen0. I noticed that no vif devices were created, only xen-br0.
>Initially, xen-br0 doesn't have an IP. I've tried setting it to the IP
>of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
>
>I changed the xend config to use the network-route rather than network,
>and when I start xend, networking doesn't go down, but xen-br0 is not
>created. I am not sure why the default network config is not working.
>Do I need to make changes to the routing tables?
>
>
I initially had this problem too, and like you, I switched to routing
until I
got everything else working and realised that routing just didn't cut it!
I'd suggest you get everything else going first.

[snip]

>disk = [ 'file:/clients/test/root_fs,sda1,w', 'file:/clients/test/swap_fs,sda2,w' ]
>
>
[snip]

># Set root device.
>root = "/dev/hda1 ro"
>
>
[snip]

Your root device seems incorrectly set. Your device export line suggests
that the line should be:
root = "/dev/sda1 ro"

Hope that helps, and good luck :)

Marcus.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: Xen questions - Swap, networking and errors [ In reply to ]

dlang at sevenl

Jul 7, 2005, 8:13 AM

Post #4 of 5 (616 views)

Permalink

M.A. Williamson wrote:
>> This is my first time trying Xen. I have installed Xen-2.0.6, on a
>> Debian Sarge server. I have a few questions that I haven't been able to
>> find answers to in the documentation or FAQs.
>
>
> OK, welcome to the community! I'll try and answer your questions. If
> there's anything you feel should be in a FAQ, feel free to get a Wiki
> account and add it at http://wiki.xensource.com.
>
>> 1)When I boot xen0, the specified amount of RAM shows up in top
>> correctly, but the swap partitions I have are not activated. When I
>> attempt to activate swap myself, I recieve the following error message:
>> xen:/# swapon /dev/hde2
>> swapon: /dev/hde2: No such device or address
>> The server has a mirrored RAID1 setup, and both MD devices are detected
>> and mounted properly. However, I am confused about why the kernel does
>> not see the hard disk partitions themselves. The disks are SATA drives,
>> and so at first I thought perhaps it didn't recongize the drives, but if
>> that were the case, I don't think either MD device would be detected
>> properly. Is there a boot option that I have not seen to address this
>
>
> Are all the drives in the machine the same? Are other partitions on hde
> in the RAID configuration.
>
> If the drives are on different controllers (e.g. a couple of PATA and a
> SATA, as on some systems) then the default kernel config may not be
> building the drivers for hde but still builds enough drivers for the MD
> devices to work.
>
>> As a side note, what is a proper amount of RAM needed for xen0? I have
>> not found any documentation really stating how much RAM is need for the
>> priveledged Xen instance. The server itself has 4GB of RAM. Should
>> xen0 have all 4GB of RAM to partition to the xenU domains?
>
>

I went back and checked /proc/partitions. Under a normal kernel, all of
the partitions show up as /dev/hde* and /dev/hdg*, but with the Xen
kernel, all of the partitions are /dev/sda* and /dev/sdb* Now that I've
found the correct device names, swap is turned on.

> RAM that you give to dom0 can't be allocated to domUs. 256MB should
> probably be enough for a slim dom0. If you use LVM snapshots that'll
> need to be larger though.
>
> You can use the "balloon driver" to reduce the size of dom0 after
> booting if you need more memory for other domains.
>
>> 2)With the default xend config, when I start xend the bridge device
>> xen-br0 is created. However, after that I lose all network activity to
>> xen0. I noticed that no vif devices were created, only xen-br0.
>> Initially, xen-br0 doesn't have an IP. I've tried setting it to the IP
>> of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
>
>
> Sounds a bit broken... What should happen is that eth0's IP is
> transferred to xen-br0.
>
> Do you have any existing bridge setup? Anything else unusual in your
> networking config?

This is a brand new server meant to run Xen. It has an Intel gigabit
NIC, but only one. I was using the default configuration from a source
install of Xen-2.0.6. There are/were not any other bridge devices
working. The thing that stood out to me was that no vif devices were
created, which from the documentation I've found, seems unusual.

>
>> I changed the xend config to use the network-route rather than network,
>> and when I start xend, networking doesn't go down, but xen-br0 is not
>> created. I am not sure why the default network config is not working.
>> Do I need to make changes to the routing tables?
>
>
> You won't get a xen-br0 if you're using the routed config - you don't
> need it. Don't know about the routing tables as I've never used the
> routed config myself.
>
> <snip>
>
>> ioctl: LOOP_SET_FD: Device or resource busy
>
> <snip>
>
>> vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
>
>
>> I am not sure why the error message LOOP_SET_FD: Device or resource busy
>> appears, since the loopback device I created the root filesystem on is
>> not mounted or being used.
>
>
> What does ls -l /dev/loop* give you?

xen:~# ls -l /dev/loop*
brw-rw---- 1 root disk 7, 0 Jun 29 16:32 /dev/loop0
brw-rw---- 1 root disk 7, 1 Jun 29 16:32 /dev/loop1
brw-rw---- 1 root disk 7, 2 Jun 29 16:32 /dev/loop2
brw-rw---- 1 root disk 7, 3 Jun 29 16:32 /dev/loop3
brw-rw---- 1 root disk 7, 4 Jun 29 16:32 /dev/loop4
brw-rw---- 1 root disk 7, 5 Jun 29 16:32 /dev/loop5
brw-rw---- 1 root disk 7, 6 Jun 29 16:32 /dev/loop6
brw-rw---- 1 root disk 7, 7 Jun 29 16:32 /dev/loop7

Thanks for the help so far. One problem taken care of. Are there
developer docs that might help explain the reason for the xenU instance
crashing?

Regards,

Dan Lang

--
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: Spam:*, Xen questions - Swap, networking and errors [ In reply to ]

dlang at sevenl

Jul 7, 2005, 10:23 AM

Post #5 of 5 (613 views)

Permalink

Marcus Brown wrote:
>
> Daniel Lang wrote:
>
>
>>1)When I boot xen0, the specified amount of RAM shows up in top
>>correctly, but the swap partitions I have are not activated. When I
>>attempt to activate swap myself, I recieve the following error message:
>> xen:/# swapon /dev/hde2
>> swapon: /dev/hde2: No such device or address
>>The server has a mirrored RAID1 setup, and both MD devices are detected
>>and mounted properly. However, I am confused about why the kernel does
>>not see the hard disk partitions themselves. The disks are SATA drives,
>>and so at first I thought perhaps it didn't recongize the drives, but if
>>that were the case, I don't think either MD device would be detected
>>properly. Is there a boot option that I have not seen to address this
>>issue?
>>
>
> Make sure the device exists with #fdisk -l /dev/hde
> I guess it's possible that your drives are now allocated a different node.
> On your original kernel, it would appear from what you're saying,
> that your SATA drives all came up as hdX.
> However, on you dom0 kernel, they are sdX.
>

Yes, I discovered this. Thank you pointing it out.

>
>>As a side note, what is a proper amount of RAM needed for xen0? I have
>>not found any documentation really stating how much RAM is need for the
>>priveledged Xen instance. The server itself has 4GB of RAM. Should
>>xen0 have all 4GB of RAM to partition to the xenU domains?
>>
>>
>
> The kernel stanza in grub's menu.lst specifies the amount of memory for
> dom0 to use, not how much it has to share. I'm not certain about the actual
> memory overheads involved in running domains, but I've found that 128MB
> is more than enough to run a couple of domains, NFS server, etc.
>
>
>>2)With the default xend config, when I start xend the bridge device
>>xen-br0 is created. However, after that I lose all network activity to
>>xen0. I noticed that no vif devices were created, only xen-br0.
>>Initially, xen-br0 doesn't have an IP. I've tried setting it to the IP
>>of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
>>
>>I changed the xend config to use the network-route rather than network,
>>and when I start xend, networking doesn't go down, but xen-br0 is not
>>created. I am not sure why the default network config is not working.
>>Do I need to make changes to the routing tables?
>>
>>
>
> I initially had this problem too, and like you, I switched to routing
> until I
> got everything else working and realised that routing just didn't cut it!
> I'd suggest you get everything else going first.
>

I've got the test domain running now, and networking for xenU doesn't
seem to work. Here is the current output from ifconfig:

xen:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0E:0C:3D:C2:7F
inet addr:204.225.94.83 Bcast:204.225.94.127
Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:121293 errors:0 dropped:0 overruns:0 frame:0
TX packets:52205 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8393755 (8.0 MiB) TX bytes:4168294 (3.9 MiB)
Base address:0xec00 Memory:defa0000-defc0000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:3494 errors:0 dropped:0 overruns:0 frame:0
TX packets:3494 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:296403 (289.4 KiB) TX bytes:296403 (289.4 KiB)

vif3.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:169.254.1.0 Bcast:169.254.255.255 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:462 (462.0 b) TX bytes:0 (0.0 b)

Is the network settings for vif3.0 the result of the not configuring the
vif variable in my xenU config? I am hesitant specifying a mac address
as I'm afraid of fubar'ing something.

> [snip]
>
>
>>disk = [ 'file:/clients/test/root_fs,sda1,w', 'file:/clients/test/swap_fs,sda2,w' ]
>>
>>
>
> [snip]
>
>
>># Set root device.
>>root = "/dev/hda1 ro"
>>
>>
>
> [snip]
>
> Your root device seems incorrectly set. Your device export line suggests
> that the line should be:
> root = "/dev/sda1 ro"
>

Aww! Yes this was it. I always seem to miss some mundane detail such as
that. Thank you for being an extra set of eyes.

>
> Hope that helps, and good luck :)
>

Thanks for the help so far.

> Marcus.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

--
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mailing List Archive

Mailing List Archive

Attached Files: