Mailing List Archive

On Mon, Jul 15, 2019 at 3:07 PM Stefan Malte Schumacher
<stefan.schumacher.netfed@gmail.com> wrote:
>
> Hello,
>
> we are currently planning to create about 40 new VMs on Debian 10 and there are
> some technology decisions we want to make before starting. We have already decided
> that we will use PVH instead of PV guests. Now we are debating how to store the VMs.
> Until now we have used the traditional approach with a LVM. A colleague showed me some of the things he can do with proxmox and qcow2 and I would like to give qcow2-images a try. We have midterm plans of migrating to a SAN and this would most likely be easier with images instead of a LVM.
>
> I have found out on my own that Xen supports qcow2 in theory, but I still have two questions:
>
> 1) How mature is Xen's support for qcow2 as compared to LVM or raw images?

For qcow2, Xen uses a mode called `qdev`, where an instance of QEMU
itself acts as a disk backend; this means that there should be
absolutely no issues with image compatibility, since it's exactly the
same code reading the image as in upstream QEMU. Functionally, it
gets tested regularly and so should have no stability issues.

I'm not sure if anyone has been investing in performance -- perhaps
Paul can comment on that.

> 2) How do I set up a new guest with qcow2? The Xen-Tools seem only to support Raw-Images
> and LVM? Surprisingly I have found very little information about both my questions on the WWW, I hope that somebody on this list can provide me with more information.

Unfortunately I'm not familiar with higher-level tools like xen-tools.
It theoretically shouldn't be difficult to extend xen-tools to build
qdisk images. Below are the commands I have in my own scripts for
creating qcow images:

* Create a disk image:

qemu-img create -f qcow2 "${image}" "${size}"M

* Mount such an image in dom0:

xl block-attach 0 ${blockspec}

Where ${blockspec} is:

"backendtype=qdev,vdev=${dev},format=qcow2,target=${image}"

Then you can use parted to create partitions, and `mount
/dev/${dev}${part}` and populate the filesystem. (Or not use
partitions if you don't want to.)

After that, the rest of the xen-tools install should Just Work.

Alternately, you could just create the disk as described in the first
step, and boot an installer.

-George

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users

> -----Original Message-----
> From: George Dunlap <dunlapg@umich.edu>
> Sent: 15 July 2019 16:36
> To: Stefan Malte Schumacher <stefan.schumacher.netfed@gmail.com>
> Cc: xen-users <xen-users@lists.xenproject.org>; Paul Durrant <Paul.Durrant@citrix.com>
> Subject: Re: [Xen-users] qcow2 and Xen
>
> On Mon, Jul 15, 2019 at 3:07 PM Stefan Malte Schumacher
> <stefan.schumacher.netfed@gmail.com> wrote:
> >
> > Hello,
> >
> > we are currently planning to create about 40 new VMs on Debian 10 and there are
> > some technology decisions we want to make before starting. We have already decided
> > that we will use PVH instead of PV guests. Now we are debating how to store the VMs.
> > Until now we have used the traditional approach with a LVM. A colleague showed me some of the things
> he can do with proxmox and qcow2 and I would like to give qcow2-images a try. We have midterm plans of
> migrating to a SAN and this would most likely be easier with images instead of a LVM.
> >
> > I have found out on my own that Xen supports qcow2 in theory, but I still have two questions:
> >
> > 1) How mature is Xen's support for qcow2 as compared to LVM or raw images?
>
> For qcow2, Xen uses a mode called `qdev`, where an instance of QEMU
> itself acts as a disk backend; this means that there should be
> absolutely no issues with image compatibility, since it's exactly the
> same code reading the image as in upstream QEMU. Functionally, it
> gets tested regularly and so should have no stability issues.
>
> I'm not sure if anyone has been investing in performance -- perhaps
> Paul can comment on that.

I and others have added some performance improvements (e.g. multi-page ring and iothread) but there are more in the pipeline (grant op batching for one) when time permits. Citrix Hypervisor is now actively using QEMU and qcow2 images so they do get a reasonable level scrutiny in a Xen environment.

Paul
_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users

Thank you very much - I couldnt have hoped for a more competent reply. One
final question: Is this still the correct way of mounting qcow2-images?
https://wiki.xen.org/wiki/Blktap2 This is from 2010 and I guess that things
might have changed in the last decade.

Yours sincerely
Sttefan Schumacher


Am Mo., 15. Juli 2019 um 17:35 Uhr schrieb George Dunlap <dunlapg@umich.edu
>:

> On Mon, Jul 15, 2019 at 3:07 PM Stefan Malte Schumacher
> <stefan.schumacher.netfed@gmail.com> wrote:
> >
> > Hello,
> >
> > we are currently planning to create about 40 new VMs on Debian 10 and
> there are
> > some technology decisions we want to make before starting. We have
> already decided
> > that we will use PVH instead of PV guests. Now we are debating how to
> store the VMs.
> > Until now we have used the traditional approach with a LVM. A colleague
> showed me some of the things he can do with proxmox and qcow2 and I would
> like to give qcow2-images a try. We have midterm plans of migrating to a
> SAN and this would most likely be easier with images instead of a LVM.
> >
> > I have found out on my own that Xen supports qcow2 in theory, but I
> still have two questions:
> >
> > 1) How mature is Xen's support for qcow2 as compared to LVM or raw
> images?
>
> For qcow2, Xen uses a mode called `qdev`, where an instance of QEMU
> itself acts as a disk backend; this means that there should be
> absolutely no issues with image compatibility, since it's exactly the
> same code reading the image as in upstream QEMU. Functionally, it
> gets tested regularly and so should have no stability issues.
>
> I'm not sure if anyone has been investing in performance -- perhaps
> Paul can comment on that.
>
> > 2) How do I set up a new guest with qcow2? The Xen-Tools seem only to
> support Raw-Images
> > and LVM? Surprisingly I have found very little information about both my
> questions on the WWW, I hope that somebody on this list can provide me with
> more information.
>
> Unfortunately I'm not familiar with higher-level tools like xen-tools.
> It theoretically shouldn't be difficult to extend xen-tools to build
> qdisk images. Below are the commands I have in my own scripts for
> creating qcow images:
>
> * Create a disk image:
>
> qemu-img create -f qcow2 "${image}" "${size}"M
>
> * Mount such an image in dom0:
>
> xl block-attach 0 ${blockspec}
>
> Where ${blockspec} is:
>
> "backendtype=qdev,vdev=${dev},format=qcow2,target=${image}"
>
> Then you can use parted to create partitions, and `mount
> /dev/${dev}${part}` and populate the filesystem. (Or not use
> partitions if you don't want to.)
>
> After that, the rest of the xen-tools install should Just Work.
>
> Alternately, you could just create the disk as described in the first
> step, and boot an installer.
>
> -George
>

On 7/16/19 2:33 PM, Stefan Malte Schumacher wrote:
> Thank you very much - I couldnt have hoped for a more competent reply.
> One final question: Is this still the correct way of mounting qcow2-images?
> https://wiki.xen.org/wiki/Blktap2 This is from 2010 and I guess that
> things might have changed in the last decade.

Yes the Wiki is outdated and the doc is rather maintained within the
upstream source now. The legacy syntax as shown in your link is still
supported so you might keep doing it this way. I am not sure how to
point to a qcow2 file with the new syntax.
https://xenbits.xen.org/docs/unstable/man/xl-disk-configuration.5.html

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users

> -----Original Message-----
> From: pierre-philipp braun <pbraun@nethence.com>
> Sent: 17 July 2019 11:43
> To: Stefan Malte Schumacher <stefan.schumacher.netfed@gmail.com>; George Dunlap <dunlapg@umich.edu>
> Cc: Paul Durrant <Paul.Durrant@citrix.com>; xen-users <xen-users@lists.xenproject.org>
> Subject: Re: [Xen-users] qcow2 and Xen
>
> On 7/16/19 2:33 PM, Stefan Malte Schumacher wrote:
> > Thank you very much - I couldnt have hoped for a more competent reply.
> > One final question: Is this still the correct way of mounting qcow2-images?
> > https://wiki.xen.org/wiki/Blktap2 This is from 2010 and I guess that
> > things might have changed in the last decade.
>
> Yes the Wiki is outdated and the doc is rather maintained within the
> upstream source now. The legacy syntax as shown in your link is still
> supported so you might keep doing it this way. I am not sure how to
> point to a qcow2 file with the new syntax.
> https://xenbits.xen.org/docs/unstable/man/xl-disk-configuration.5.html

FWIW the form I use is:

'format=qcow2,vdev=<vdev>,access=rw,target=<path to qcow2 file>'

Paul

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users

Hello,

I went the way of least effort and created a pvh-guest as raw-image with
xen-tools and then converted it with qemu, which took about 30 seconds. I
am going to automatize this step in ansible, which already runs
xen-create-images
based on the given hostvars. Unfortunately the domain does not start. I get
the following error message:

xl create dzb327.netfed.de.cfg
Parsing config from dzb327.netfed.de.cfg
libxl: error: libxl_device.c:1418:libxl__wait_for_backend: Backend
/local/domain/0/backend/qdisk/0/51712 not ready
libxl: error: libxl_bootloader.c:417:bootloader_disk_attached_cb: Domain
114:failed to attach local disk for bootloader execution
libxl: error: libxl_bootloader.c:283:bootloader_local_detached_cb: Domain
114:unable to detach locally attached disk
libxl: error: libxl_create.c:1267:domcreate_rebuild_done: Domain 114:cannot
(re-)build domain: -3
libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain
114:Non-existant domain
libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 114:Unable
to destroy guest
libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 114:Destruction
of domain failed


These are the relevant parts of my dzb327.netfed.de.cfg:

bootloader = 'pygrub'

type = 'pvh'
vcpus = '2'
memory = '4096'
maxmem = '4096'


#
# Disk device(s).
#
root = '/dev/xvda2 ro'
disk = [
'tap2:qcow2:/home/xen/domains/
dzb327.netfed.de/disk.img,xvda2,w',
'tap2:qcow2:/home/xen/domains/
dzb327.netfed.de/swap.img,xvda1,w',
]
#

How do I do this right? I have found dozen of hits for booting xen with
raw-images converted to qcow2, so it seems I am not trying anthing out of
the ordinary here.

Yours sincerely
Stefan

On 7/17/19 1:18 PM, Stefan Malte Schumacher wrote:
> Hello,
>
> I went the way of least effort and created a pvh-guest as raw-image with
> xen-tools and then converted it with qemu, which took about 30 seconds.
> I am going to automatize this step in ansible, which already runs
> xen-create-images
> based on the given hostvars. Unfortunately the domain does not start. I
> get the following error message:
>
> xl create dzb327.netfed.de.cfg
> Parsing config from dzb327.netfed.de.cfg
> libxl: error: libxl_device.c:1418:libxl__wait_for_backend: Backend
> /local/domain/0/backend/qdisk/0/51712 not ready
> libxl: error: libxl_bootloader.c:417:bootloader_disk_attached_cb: Domain
> 114:failed to attach local disk for bootloader execution
> libxl: error: libxl_bootloader.c:283:bootloader_local_detached_cb:
> Domain 114:unable to detach locally attached disk
> libxl: error: libxl_create.c:1267:domcreate_rebuild_done: Domain
> 114:cannot (re-)build domain: -3
> libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain
> 114:Non-existant domain
> libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain
> 114:Unable to destroy guest
> libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain
> 114:Destruction of domain failed
>
>
> These are the relevant parts of my dzb327.netfed.de.cfg:
>
> bootloader = 'pygrub'

What is the guest OS? Also Debian 10? With the 4.19 kernel in Debian 10,
you can use grub2 in combination with PVH.

You need the grub-xen-host package on the dom0, and in the domU you need
the grub-xen package. No actual grub install / boot sector etc is
needed, only the config file (remember, it's PVH, not HVM).

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776450#95

kernel = "/usr/lib/grub-xen/grub-i386-xen_pvh.bin"
type = "pvh"

> type = 'pvh'
> vcpus       = '2'
> memory      = '4096'
> maxmem      = '4096'
>
>
> #
> #  Disk device(s).
> #
> root        = '/dev/xvda2 ro'
> disk        = [.
>                  
> 'tap2:qcow2:/home/xen/domains/dzb327.netfed.de/disk.img,xvda2,w
> <http://dzb327.netfed.de/disk.img,xvda2,w>',
>                  
> 'tap2:qcow2:/home/xen/domains/dzb327.netfed.de/swap.img,xvda1,w
> <http://dzb327.netfed.de/swap.img,xvda1,w>',
>               ]
> #

Is there a specific reason you're using xvda1, xvda2 instead of just
xvda, xvdb?

>
> How do I do this right? I have found dozen of hits for booting xen with
> raw-images converted to qcow2, so it seems I am not trying anthing out
> of the ordinary here.

Hans


_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users

>
> What is the guest OS? Also Debian 10? With the 4.19 kernel in Debian 10,
> you can use grub2 in combination with PVH.
>
> it is Debian 10. But do you think that pygrub is the reason why the
machine is not booting? After all, it works very well
when starting it as a raw image.


> You need the grub-xen-host package on the dom0, and in the domU you need
> the grub-xen package. No actual grub install / boot sector etc is
> needed, only the config file (remember, it's PVH, not HVM).
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776450#95
>
> I'll have a look at that tomorrow.


> Is there a specific reason you're using xvda1, xvda2 instead of just
> xvda, xvdb?
>

Thats the way xen-tools created them. I do not know if this is a default
setting or a change made by the admin before me.

Yours sincerely
Stefan

Hello,

is there nobody on this list who is actually using qcow and could provide
me with a working guest.cfg ? I am using Debian 10 and Xen 4.11. This -
https://xenbits.xen.org/docs/unstable/man/xl-disk-configuration.5.html -
is unfortunately somewhat over my head, xen-wise.

Yours sincerely
Stefan

Am Mi., 17. Juli 2019 um 17:48 Uhr schrieb Stefan Malte Schumacher <
stefan.schumacher.netfed@gmail.com>:

>
>
>> What is the guest OS? Also Debian 10? With the 4.19 kernel in Debian 10,
>> you can use grub2 in combination with PVH.
>>
>> it is Debian 10. But do you think that pygrub is the reason why the
> machine is not booting? After all, it works very well
> when starting it as a raw image.
>
>
>> You need the grub-xen-host package on the dom0, and in the domU you need
>> the grub-xen package. No actual grub install / boot sector etc is
>> needed, only the config file (remember, it's PVH, not HVM).
>>
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776450#95
>>
>> I'll have a look at that tomorrow.
>
>
>> Is there a specific reason you're using xvda1, xvda2 instead of just
>> xvda, xvdb?
>>
>
> Thats the way xen-tools created them. I do not know if this is a default
> setting or a change made by the admin before me.
>
> Yours sincerely
> Stefan
>
>
>