Mailing List Archive

Booting Xen on qemu-arm
Hello,

I am trying to boot xen on qemu arm. I was able to boot Linux kernel
with u-boot:

~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -cpu cortex-a15 -machine
virt -m 256M -bios u-boot.bin -device
loader,file=zImage,force-raw=on,addr=0x41000000 -nographic -no-reboot
-chardev
socket,id=qemu-monitor,host=localhost,port=7777,server,nowait,telnet
-mon qemu-monitor,mode=readline

Then, in u-boot:

setenv bootargs root=/dev/ram rw earlyprintk=serial,ttyAMA0 console=ttyAMA0

bootz 0x41000000 - 0x40000000

So, I use a device tree from this machine's default location - 0x40000000

After that, I produced an image:

mkimage -A arm -T kernel -a 0x46000000 -e 0x46000000 -C none -d
"xen/xen" xen-uImage

And tried to launch it:

 ~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -cpu cortex-a15 -machine
virt -m 512M -bios u-boot.bin -device
loader,file=xen-uImage,force-raw=on,addr=0x41000000 -device
loader,file=zImage,addr=0x45000000 -nographic -no-reboot -chardev
socket,id=qemu-monitor,host=localhost,port=7777,server,nowait,telnet
-mon qemu-monitor,mode=readline

So, I put xen image to 0x41000000 and Linux zImage to 0x45000000. I want
to use qemu's default device tree in 0x40000000.

In u-boot:

fdt addr 0x40000000
fdt resize

fdt set /chosen \#address-cells <1>
fdt set /chosen \#size-cells <1>
  
fdt set /chosen xen,xen-bootargs "console=dtuart dtuart="/pl011@9000000""
fdt mknod /chosen module@0
fdt set /chosen/module@0 compatible "xen,linux-zimage"
"xen,multiboot-module"
fdt set /chosen/module@0 reg <0x45000000 0x853200 >
fdt set /chosen/module@0 bootargs "earlyprintk=serial,ttyAMA0
console=ttyAMA0,115200n8 earlycon=xenboot"

bootm 0x41000000 - 0x40000000

And I get the output:

## Booting kernel from Legacy Image at 41000000 ...
   Image Name:  
   Created:      2019-06-02  10:16:13 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    753672 Bytes = 736 KiB
   Load Address: 46000000
   Entry Point:  46000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 40000000
   Booting using the fdt blob at 0x40000000
   Loading Kernel Image ... OK
   reserving fdt memory region: addr=40000000 size=2000
   Using Device Tree in place at 40000000, end 40004fff

Starting kernel ...

And I have no output after that.

I don't understand several things:

Where should I put xen-uImage, kernel and fdt? I assume they are loaded
to ram to the specified addresses (ram is from 0x40000000). Can I use a
default QEMU device tree in 0x40000000 or should I load the one
generated by u-boot with the command:

~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -machine virt,gic_version=3 -machine virtualization=true -cpu cortex-a15 -machine type=virt -m 512 -display none -machine dumpdtb=virt.dtb

Also, we have created a node /chosen/module@0. Where can I find its description? What is it?

--
Regards, Denis Obrezkov
Re: Booting Xen on qemu-arm [ In reply to ]
On 02/06/2019 14:55, Denis Obrezkov wrote:
> Hello,

Hi Denis,

> I am trying to boot xen on qemu arm. I was able to boot Linux kernel
> with u-boot:
>
> ~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -cpu cortex-a15 -machine
> virt -m 256M -bios u-boot.bin -device
> loader,file=zImage,force-raw=on,addr=0x41000000 -nographic -no-reboot
> -chardev
> socket,id=qemu-monitor,host=localhost,port=7777,server,nowait,telnet
> -mon qemu-monitor,mode=readline
>
> Then, in u-boot:
>
> setenv bootargs root=/dev/ram rw earlyprintk=serial,ttyAMA0 console=ttyAMA0
>
> bootz 0x41000000 - 0x40000000
>
> So, I use a device tree from this machine's default location - 0x40000000
>
> After that, I produced an image:
>
> mkimage -A arm -T kernel -a 0x46000000 -e 0x46000000 -C none -d
> "xen/xen" xen-uImage

You don't need to produce a uImage here. Xen is by default supporting the Linux
boot protocol, so you could use bootz.

>
> And tried to launch it:
>
>  ~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -cpu cortex-a15 -machine
> virt -m 512M -bios u-boot.bin -device
> loader,file=xen-uImage,force-raw=on,addr=0x41000000 -device
> loader,file=zImage,addr=0x45000000 -nographic -no-reboot -chardev
> socket,id=qemu-monitor,host=localhost,port=7777,server,nowait,telnet
> -mon qemu-monitor,mode=readline
>
> So, I put xen image to 0x41000000 and Linux zImage to 0x45000000. I want
> to use qemu's default device tree in 0x40000000.
>
> In u-boot:
>
> fdt addr 0x40000000
> fdt resize
>
> fdt set /chosen \#address-cells <1>
> fdt set /chosen \#size-cells <1>
>
> fdt set /chosen xen,xen-bootargs "console=dtuart dtuart="/pl011@9000000""
> fdt mknod /chosen module@0
> fdt set /chosen/module@0 compatible "xen,linux-zimage"
> "xen,multiboot-module"
> fdt set /chosen/module@0 reg <0x45000000 0x853200 >
> fdt set /chosen/module@0 bootargs "earlyprintk=serial,ttyAMA0
> console=ttyAMA0,115200n8 earlycon=xenboot"
>
> bootm 0x41000000 - 0x40000000
>
> And I get the output:
>
> ## Booting kernel from Legacy Image at 41000000 ...
>    Image Name:
>    Created:      2019-06-02  10:16:13 UTC
>    Image Type:   ARM Linux Kernel Image (uncompressed)
>    Data Size:    753672 Bytes = 736 KiB
>    Load Address: 46000000
>    Entry Point:  46000000
>    Verifying Checksum ... OK
> ## Flattened Device Tree blob at 40000000
>    Booting using the fdt blob at 0x40000000
>    Loading Kernel Image ... OK
>    reserving fdt memory region: addr=40000000 size=2000
>    Using Device Tree in place at 40000000, end 40004fff
>
> Starting kernel ...
>
> And I have no output after that.

This maybe because Xen crashed before the console is setup. You can compile Xen
with earlyprintk.

Can you provide the command line you use to build Xen?

>
> I don't understand several things:
>
> Where should I put xen-uImage, kernel and fdt? I assume they are loaded
> to ram to the specified addresses (ram is from 0x40000000).

You can put the blob wherever you want in the RAM as long as it does not clash
with other things (e.g U-boot, DT).

> Can I use a
> default QEMU device tree in 0x40000000 or should I load the one
> generated by u-boot with the command:

fdt addr 0x40000000 will load the device tree and the subsequent one will
directly modify it.

>
> ~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -machine virt,gic_version=3 -machine virtualization=true -cpu cortex-a15 -machine type=virt -m 512 -display none -machine dumpdtb=virt.dtb
>
> Also, we have created a node /chosen/module@0. Where can I find its description? What is it?

You can read more about the description in [1]. In short, it used to to tell Xen
where any modules (e.g Dom0 kernel, Dom0 Initramfs,...) are loaded in memory.

Cheers,

[1] https://xenbits.xen.org/docs/unstable/misc/arm/device-tree/booting.txt

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Booting Xen on qemu-arm [ In reply to ]
Hi Julien,
> This maybe because Xen crashed before the console is setup. You can
> compile Xen with earlyprintk.
>
> Can you provide the command line you use to build Xen?
>

I've just built Xen with debugging on and found out the reason why it
didn't work:

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- CPU doesn't support the virtualization extensions -
- Boot failed -

My command line was:

~/bin/qemu-3.1.0/arm-softmmu/qemu-system-arm -cpu cortex-a15 -machine
virt -m 512M -bios u-boot.bin -device
loader,file=xen-uImage,force-raw=on,addr=0x46000000 -device
loader,file=zImage,addr=0x45000000 -nographic -no-reboot -chardev
socket,id=qemu-monitor,host=localhost,port=7777,server,nowait,telnet
-mon qemu-monitor,mode=readline

I modified if to:

-machine virt,gic_version=3 -machine virtualization=true

so, now I get further:

- UART enabled -
- CPU 00000000 booting -
- Xen starting in Hyp mode -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000000040000000 - 000000005fffffff
(XEN)
(XEN) MODULE[0]: 0000000040000000 - 0000000040002000 Device Tree
(XEN) MODULE[1]: 0000000045000000 - 0000000045853200 Kernel     
(XEN)  RESVD[0]: 0000000040000000 - 0000000040002000
(XEN)
(XEN) CMDLINE[0000000045000000]:chosen earlyprintk=serial,ttyAMA0
console=ttyAMA0,115200n8 earlycon=xenboot
(XEN)
(XEN) Command line: console=dtuart dtuart=/pl011@9000000
(XEN) Xen heap: 000000005e000000-0000000060000000 (8192 pages)
(XEN) Dom heap: 122880 pages
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Unable to find compatible GIC in the device tree
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

Though, there is a node from my device tree:

intc@8000000 {
        phandle = <0x00008001>;
        interrupts = <0x00000001 0x00000009 0x00000004>;
        reg = <0x00000000 0x08000000 0x00000000 0x00010000 0x00000000
0x080a0000 0x00000000 0x00f60000>;
        #redistributor-regions = <0x00000001>;
        compatible = "arm,gic-v3";
        ranges;
        #size-cells = <0x00000002>;
        #address-cells = <0x00000002>;
        interrupt-controller;
        #interrupt-cells = <0x00000003>;
    };

I trued to boot with gic_version=2, but also stuck:

...

(XEN) GICv2: 288 lines, 1 cpu (IID 00000000).
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler

My guess that here might be a problem with Dom0.


--
Regards, Denis Obrezkov
Re: Booting Xen on qemu-arm [ In reply to ]
On 03/06/2019 13:22, Denis Obrezkov wrote:
> Hi Julien,
>> This maybe because Xen crashed before the console is setup. You can
>> compile Xen with earlyprintk.
>>
>> Can you provide the command line you use to build Xen?
>>
> - UART enabled -
> - CPU 00000000 booting -
> - Xen starting in Hyp mode -
> - Zero BSS -
> - Setting up control registers -
> - Turning on paging -
> - Ready -
> (XEN) Checking for initrd in /chosen
> (XEN) RAM: 0000000040000000 - 000000005fffffff
> (XEN)
> (XEN) MODULE[0]: 0000000040000000 - 0000000040002000 Device Tree
> (XEN) MODULE[1]: 0000000045000000 - 0000000045853200 Kernel
> (XEN)  RESVD[0]: 0000000040000000 - 0000000040002000
> (XEN)
> (XEN) CMDLINE[0000000045000000]:chosen earlyprintk=serial,ttyAMA0
> console=ttyAMA0,115200n8 earlycon=xenboot
> (XEN)
> (XEN) Command line: console=dtuart dtuart=/pl011@9000000
> (XEN) Xen heap: 000000005e000000-0000000060000000 (8192 pages)
> (XEN) Dom heap: 122880 pages
> (XEN) Domain heap initialised
> (XEN) Booting using Device Tree
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Unable to find compatible GIC in the device tree
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
> Though, there is a node from my device tree:
>
> intc@8000000 {
>         phandle = <0x00008001>;
>         interrupts = <0x00000001 0x00000009 0x00000004>;
>         reg = <0x00000000 0x08000000 0x00000000 0x00010000 0x00000000
> 0x080a0000 0x00000000 0x00f60000>;
>         #redistributor-regions = <0x00000001>;
>         compatible = "arm,gic-v3";
>         ranges;
>         #size-cells = <0x00000002>;
>         #address-cells = <0x00000002>;
>         interrupt-controller;
>         #interrupt-cells = <0x00000003>;
>     };

Which version of Xen are you using? Also what is the content of xen/.config?

>
> I trued to boot with gic_version=2, but also stuck:

I am not entirely sure if QEMU supports GICv2 virtualization. Stefano, do you
remember?

>
> ...
>
> (XEN) GICv2: 288 lines, 1 cpu (IID 00000000).
> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
> (XEN) Initializing Credit2 scheduler
>
> My guess that here might be a problem with Dom0.

It is too early for having a problem with Dom0. It is most likely due to the
configuration of your GIC.

Cheers,

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Booting Xen on qemu-arm [ In reply to ]
> Which version of Xen are you using? Also what is the content of
> xen/.config?
>
I use xen-4.12.0.

I attached the .config.

>>
>> I trued to boot with gic_version=2, but also stuck:
>
> I am not entirely sure if QEMU supports GICv2 virtualization. Stefano,
> do you remember?

This is an interrupt controller node in fdt generated by qemu:

    intc@8000000 {
        phandle = <0x00008001>;
        interrupts = <0x00000001 0x00000009 0x00000004>;
        reg = <0x00000000 0x08000000 0x00000000 0x00010000 0x00000000
0x08010000 0x00000000 0x00010000 0x00000000 0x08030000 0x00000000
0x00010000 0x00000000 0x08040000 0x00000000 0x00010000>;
        compatible = "arm,cortex-a15-gic";
        ranges;
        #size-cells = <0x00000002>;
        #address-cells = <0x00000002>;
        interrupt-controller;
        #interrupt-cells = <0x00000003>;
        v2m@8020000 {
            phandle = <0x00008002>;
            reg = <0x00000000 0x08020000 0x00000000 0x00001000>;
            msi-controller;
            compatible = "arm,gic-v2m-frame";
        };
    };

>
>>
>> ...
>>
>> (XEN) GICv2: 288 lines, 1 cpu (IID 00000000).
>> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
>> (XEN) Initializing Credit2 scheduler
>>
>> My guess that here might be a problem with Dom0.
>
> It is too early for having a problem with Dom0. It is most likely due
> to the configuration of your GIC.
>
> Cheers,
>
--
Regards, Denis Obrezkov
Re: Booting Xen on qemu-arm [ In reply to ]
Hi,

On 6/3/19 8:19 PM, Denis Obrezkov wrote:
>
>> Which version of Xen are you using? Also what is the content of
>> xen/.config?
>>
> I use xen-4.12.0.
>
> I attached the .config.

Oh, you are building for Arm32 bit. Sorry I didn't realize it before.
Xen does not support GICv3 for 32-bit.

Also, I am not sure that virtualization is fully supported in QEMU for
32-bit. At least I haven't tested myself.

You may want to ask the QEMU folks.

Cheers,

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users