Mailing List Archive: Colibri imx8qxp: Missing kernel boot module

Colibri imx8qxp: Missing kernel boot module

Aug 10, 2020, 8:32 AM

Post #1 of 15 (2013 views)

Hello xen-users,

we are trying to get Xen running on a Toradex "Colibri iMX8X" module (see
[1] at the bottom of this email), which features the iMX8 QXP prozessor.
We found that NXP has a Xen reference implementation [2][3] for their MEK
Module and tried to port that to the Toradex module.

When booting via the bootscript [4], which is unaltered except for the
"dom0fdt_file" and "xenhyper_bootargs" variables, we get the following
error:

[...] (See [5] for complete bootlog)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Missing kernel boot module?
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up DOM0 guest OS
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

For Dom0 we took the linux-toradex kernel source, ran "make xenconfig" [6],
which should add Xen-support and rebuilt the Image with "make Image".
For dom0 DTB [7] we copied the dom0 DTS NXP uses for their MEK-Module [8]
and only adjusted the "bootargs" parameter.
This Linux Image was able to run after we rebuilt and booted it without Xen.

Question 1:
The xen dokumentation always states, that the Dom0- and DomU- Images have to
be in zImage format, but according to [9] arm64 doesn't use zImage and [10]
says the arm64/Image is equivalent to zImage.
Can we assume that our arm64/Image should work, if everything else was fine?

Question 2:
Our Xen was build with imx-xen repository [11] as source via NXP's BSP
according to [1] with MACHINE=imx8qxpmek.
Can we use the resulting Xen Hypervisor-Image for the Toradex IMX8X Module,
as it uses the same Processor (i.MX 8 QXP)?

Question 3:
If the answers to Question 1 and Question 2 are yes, then what else could we
be missing?

[1]
https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-imx-8x
[2] https://www.nxp.com/docs/en/user-guide/IMX_YOCTO_PROJECT_USERS_GUIDE.pdf
[3] https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf

Bootscript:
[4] https://pastebin.com/dcj0K3Xb

Complete Bootlog:
[5] https://pastebin.com/eLpZvd29

Xenconfig according to Xen Wiki
[6]
https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_latest_
Linux_kernel_support

fsl-imx8qxp-colibri-eval-v3-dom0.dts:
[7] https://pastebin.com/f4dGCYFf

fsl-imx8qxp-mek-dom0.dts:
[8] https://pastebin.com/c3Ukc6mZ

No arm64/zImage:
[9] https://community.nxp.com/thread/482861#comment-1051436

arm64/Image is equivalent to zImage:
[10] http://markmail.org/message/advnw6lk5rcfyabp

iMX-XEN Repo:
[11]
https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35_1.1.0

Thank you very much for your reading this mail and helping with our issue!

Best regards,

Daniel

Re: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

julien at xen

Aug 14, 2020, 12:46 PM

Post #2 of 15 (2012 views)

Permalink

On 10/08/2020 16:32, Daniel Wagner2 wrote:
> Hello xen-users,

Hello,

> we are trying to get Xen running on a Toradex "Colibri iMX8X" module (see
> [1] at the bottom of this email), which features the iMX8 QXP prozessor.
> We found that NXP has a Xen reference implementation [2][3] for their MEK
> Module and tried to port that to the Toradex module.
>
> When booting via the bootscript [4], which is unaltered except for the
> "dom0fdt_file" and "xenhyper_bootargs" variables, we get the following
> error:
>
> [...] (See [5] for complete bootlog)
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Missing kernel boot module?
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Could not set up DOM0 guest OS
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
> For Dom0 we took the linux-toradex kernel source, ran "make xenconfig" [6],
> which should add Xen-support and rebuilt the Image with "make Image".
> For dom0 DTB [7] we copied the dom0 DTS NXP uses for their MEK-Module [8]
> and only adjusted the "bootargs" parameter.
> This Linux Image was able to run after we rebuilt and booted it without Xen.

Which Device-Tree did you for boot Linux without Xen?

>
> Question 1:
> The xen dokumentation always states, that the Dom0- and DomU- Images have to
> be in zImage format, but according to [9] arm64 doesn't use zImage and [10]
> says the arm64/Image is equivalent to zImage.
> Can we assume that our arm64/Image should work, if everything else was fine?

This is a mistake in the documentation. Xen is able to load arm64/Image.

Can you give me a pointer to the documentation you are using? I will
update it.

>
> Question 2:
> Our Xen was build with imx-xen repository [11] as source via NXP's BSP
> according to [1] with MACHINE=imx8qxpmek.
> Can we use the resulting Xen Hypervisor-Image for the Toradex IMX8X Module,
> as it uses the same Processor (i.MX 8 QXP)?

I am afraid this tree is not maintained by Xen Project but, IIRC, NXP. I
have CCed one person that may be able to help you.

>
> Question 3:
> If the answers to Question 1 and Question 2 are yes, then what else could we
> be missing?
>
>
>
> [1]
> https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-imx-8x
> [2] https://www.nxp.com/docs/en/user-guide/IMX_YOCTO_PROJECT_USERS_GUIDE.pdf
> [3] https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
>
> Bootscript:
> [4] https://pastebin.com/dcj0K3Xb
>
> Complete Bootlog:
> [5] https://pastebin.com/eLpZvd29
>
> Xenconfig according to Xen Wiki
> [6]
> https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_latest_
> Linux_kernel_support
>
> fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> [7] https://pastebin.com/f4dGCYFf
>
> fsl-imx8qxp-mek-dom0.dts:
> [8] https://pastebin.com/c3Ukc6mZ
>
> No arm64/zImage:
> [9] https://community.nxp.com/thread/482861#comment-1051436
>
> arm64/Image is equivalent to zImage:
> [10] http://markmail.org/message/advnw6lk5rcfyabp
>
> iMX-XEN Repo:
> [11]
> https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35_1.1.0
>
>
>
> Thank you very much for your reading this mail and helping with our issue!
>
> Best regards,

Best regards,

--
Julien Grall

AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Aug 17, 2020, 2:40 AM

Post #3 of 15 (2011 views)

Permalink

> -----Urspr?ngliche Nachricht-----
> Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im Auftrag von
Julien
> Grall
> Gesendet: Freitag, 14. August 2020 21:46
> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
> Betreff: Re: Colibri imx8qxp: Missing kernel boot module
>
> On 10/08/2020 16:32, Daniel Wagner2 wrote:
> > Hello xen-users,
>
> Hello,
>
> > we are trying to get Xen running on a Toradex "Colibri iMX8X" module
> > (see [1] at the bottom of this email), which features the iMX8 QXP
prozessor.
> > We found that NXP has a Xen reference implementation [2][3] for their
> > MEK Module and tried to port that to the Toradex module.
> >
> > When booting via the bootscript [4], which is unaltered except for the
> > "dom0fdt_file" and "xenhyper_bootargs" variables, we get the following
> > error:
> >
> > [...] (See [5] for complete bootlog)
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Missing kernel boot module?
> > (XEN)
> > (XEN) ****************************************
> > (XEN) Panic on CPU 0:
> > (XEN) Could not set up DOM0 guest OS
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Reboot in five seconds...
> >
> > For Dom0 we took the linux-toradex kernel source, ran "make xenconfig"
> > [6], which should add Xen-support and rebuilt the Image with "make
Image".
> > For dom0 DTB [7] we copied the dom0 DTS NXP uses for their MEK-Module
> > [8] and only adjusted the "bootargs" parameter.
> > This Linux Image was able to run after we rebuilt and booted it without
Xen.
>
> Which Device-Tree did you for boot Linux without Xen?

Used the same Device-Tree Binary that we want to use with Xen which is
described in [7]. In [7] you can see rows 23 and 24, which include the
Devicetree as it is supplied by Toradex.
To boot linux with our dom0 DTB, the u-boot variable fdt_file was changed
from fsl-imx8qxp-colibri-eval-v3.dtb to
fsl-imx8qxp-colibri-eval-v3-dom0.dtb.

>
> >
> > Question 1:
> > The xen dokumentation always states, that the Dom0- and DomU- Images
> > have to be in zImage format, but according to [9] arm64 doesn't use
> > zImage and [10] says the arm64/Image is equivalent to zImage.
> > Can we assume that our arm64/Image should work, if everything else was
> fine?
>
> This is a mistake in the documentation. Xen is able to load arm64/Image.

Thanks for the confirmation!

>
> Can you give me a pointer to the documentation you are using? I will
update it.
>

' where "/root/image" is a Linux zImage.'
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#DomU
_kernel_and_DTS

> >
> > Question 2:
> > Our Xen was build with imx-xen repository [11] as source via NXP's BSP
> > according to [1] with MACHINE=imx8qxpmek.
> > Can we use the resulting Xen Hypervisor-Image for the Toradex IMX8X
> > Module, as it uses the same Processor (i.MX 8 QXP)?
>
> I am afraid this tree is not maintained by Xen Project but, IIRC, NXP. I
have CCed
> one person that may be able to help you.

Thanks for connecting!

> >
> > Question 3:
> > If the answers to Question 1 and Question 2 are yes, then what else
> > could we be missing?
> >
> >
> >
> > [1]
> > https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-
> > imx-8x [2]
> > https://www.nxp.com/docs/en/user-
> guide/IMX_YOCTO_PROJECT_USERS_GUIDE.p
> > df [3]
> > https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
> >
> > Bootscript:
> > [4] https://pastebin.com/dcj0K3Xb
> >
> > Complete Bootlog:
> > [5] https://pastebin.com/eLpZvd29
> >
> > Xenconfig according to Xen Wiki
> > [6]
> > https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_l
> > atest_
> > Linux_kernel_support
> >
> > fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> > [7] https://pastebin.com/f4dGCYFf
> >
> > fsl-imx8qxp-mek-dom0.dts:
> > [8] https://pastebin.com/c3Ukc6mZ
> >
> > No arm64/zImage:
> > [9] https://community.nxp.com/thread/482861#comment-1051436
> >
> > arm64/Image is equivalent to zImage:
> > [10] http://markmail.org/message/advnw6lk5rcfyabp
> >
> > iMX-XEN Repo:
> > [11]
> > https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35
> > _1.1.0
> >
> >
> >
> > Thank you very much for your reading this mail and helping with our
issue!
> >
> > Best regards,
>
> Best regards,
>
> --
> Julien Grall

Best regards,

Daniel Wagner

RE: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

peng.fan at nxp

Aug 17, 2020, 2:47 AM

Post #4 of 15 (2011 views)

Permalink

> Subject: AW: Colibri imx8qxp: Missing kernel boot module
>
>
>
> > -----Urspr?ngliche Nachricht-----
> > Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im Auftrag von
> Julien
> > Grall
> > Gesendet: Freitag, 14. August 2020 21:46
> > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> > users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
> > Betreff: Re: Colibri imx8qxp: Missing kernel boot module
> >
> > On 10/08/2020 16:32, Daniel Wagner2 wrote:
> > > Hello xen-users,
> >
> > Hello,
> >
> > > we are trying to get Xen running on a Toradex "Colibri iMX8X" module
> > > (see [1] at the bottom of this email), which features the iMX8 QXP
> prozessor.
> > > We found that NXP has a Xen reference implementation [2][3] for their
> > > MEK Module and tried to port that to the Toradex module.
> > >
> > > When booting via the bootscript [4], which is unaltered except for the
> > > "dom0fdt_file" and "xenhyper_bootargs" variables, we get the following
> > > error:
> > >
> > > [...] (See [5] for complete bootlog)
> > > (XEN) *** LOADING DOMAIN 0 ***
> > > (XEN) Missing kernel boot module?
> > > (XEN)
> > > (XEN) ****************************************
> > > (XEN) Panic on CPU 0:
> > > (XEN) Could not set up DOM0 guest OS
> > > (XEN) ****************************************
> > > (XEN)
> > > (XEN) Reboot in five seconds...
> > >
> > > For Dom0 we took the linux-toradex kernel source, ran "make xenconfig"
> > > [6], which should add Xen-support and rebuilt the Image with "make
> Image".
> > > For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> MEK-Module
> > > [8] and only adjusted the "bootargs" parameter.
> > > This Linux Image was able to run after we rebuilt and booted it without
> Xen.
> >
> > Which Device-Tree did you for boot Linux without Xen?
>
> Used the same Device-Tree Binary that we want to use with Xen which is
> described in [7]. In [7] you can see rows 23 and 24, which include the
> Devicetree as it is supplied by Toradex.
> To boot linux with our dom0 DTB, the u-boot variable fdt_file was changed
> from fsl-imx8qxp-colibri-eval-v3.dtb to
> fsl-imx8qxp-colibri-eval-v3-dom0.dtb.

Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?

Regards,
Peng.

>
> >
> > >
> > > Question 1:
> > > The xen dokumentation always states, that the Dom0- and DomU- Images
> > > have to be in zImage format, but according to [9] arm64 doesn't use
> > > zImage and [10] says the arm64/Image is equivalent to zImage.
> > > Can we assume that our arm64/Image should work, if everything else was
> > fine?
> >
> > This is a mistake in the documentation. Xen is able to load arm64/Image.
>
> Thanks for the confirmation!
>
> >
> > Can you give me a pointer to the documentation you are using? I will
> update it.
> >
>
> ' where "/root/image" is a Linux zImage.'
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#D
> omU
> _kernel_and_DTS
>
> > >
> > > Question 2:
> > > Our Xen was build with imx-xen repository [11] as source via NXP's BSP
> > > according to [1] with MACHINE=imx8qxpmek.
> > > Can we use the resulting Xen Hypervisor-Image for the Toradex IMX8X
> > > Module, as it uses the same Processor (i.MX 8 QXP)?
> >
> > I am afraid this tree is not maintained by Xen Project but, IIRC, NXP. I
> have CCed
> > one person that may be able to help you.
>
> Thanks for connecting!
>
> > >
> > > Question 3:
> > > If the answers to Question 1 and Question 2 are yes, then what else
> > > could we be missing?
> > >
> > >
> > >
> > > [1]
> > >
> https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-
> > > imx-8x [2]
> > > https://www.nxp.com/docs/en/user-
> > guide/IMX_YOCTO_PROJECT_USERS_GUIDE.p
> > > df [3]
> > >
> https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
> > >
> > > Bootscript:
> > > [4] https://pastebin.com/dcj0K3Xb
> > >
> > > Complete Bootlog:
> > > [5] https://pastebin.com/eLpZvd29
> > >
> > > Xenconfig according to Xen Wiki
> > > [6]
> > >
> https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_l
> > > atest_
> > > Linux_kernel_support
> > >
> > > fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> > > [7] https://pastebin.com/f4dGCYFf
> > >
> > > fsl-imx8qxp-mek-dom0.dts:
> > > [8] https://pastebin.com/c3Ukc6mZ
> > >
> > > No arm64/zImage:
> > > [9] https://community.nxp.com/thread/482861#comment-1051436
> > >
> > > arm64/Image is equivalent to zImage:
> > > [10] http://markmail.org/message/advnw6lk5rcfyabp
> > >
> > > iMX-XEN Repo:
> > > [11]
> > >
> https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35
> > > _1.1.0
> > >
> > >
> > >
> > > Thank you very much for your reading this mail and helping with our
> issue!
> > >
> > > Best regards,
> >
> > Best regards,
> >
> > --
> > Julien Grall
>
> Best regards,
>
> Daniel Wagner

AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Aug 17, 2020, 2:54 AM

Post #5 of 15 (2011 views)

Permalink

> -----Urspr?ngliche Nachricht-----
> Von: Peng Fan <peng.fan@nxp.com>
> Gesendet: Montag, 17. August 2020 11:47
> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; Julien Grall
> <julien@xen.org>; xen-users@lists.xenproject.org
> Betreff: RE: Colibri imx8qxp: Missing kernel boot module
>
> > Subject: AW: Colibri imx8qxp: Missing kernel boot module
> >
> >
> >
> > > -----Urspr?ngliche Nachricht-----
> > > Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im Auftrag
> > > von
> > Julien
> > > Grall
> > > Gesendet: Freitag, 14. August 2020 21:46
> > > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> > > users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
> > > Betreff: Re: Colibri imx8qxp: Missing kernel boot module
> > >
> > > On 10/08/2020 16:32, Daniel Wagner2 wrote:
> > > > Hello xen-users,
> > >
> > > Hello,
> > >
> > > > we are trying to get Xen running on a Toradex "Colibri iMX8X"
> > > > module (see [1] at the bottom of this email), which features the
> > > > iMX8 QXP
> > prozessor.
> > > > We found that NXP has a Xen reference implementation [2][3] for
> > > > their MEK Module and tried to port that to the Toradex module.
> > > >
> > > > When booting via the bootscript [4], which is unaltered except for
> > > > the "dom0fdt_file" and "xenhyper_bootargs" variables, we get the
> > > > following
> > > > error:
> > > >
> > > > [...] (See [5] for complete bootlog)
> > > > (XEN) *** LOADING DOMAIN 0 ***
> > > > (XEN) Missing kernel boot module?
> > > > (XEN)
> > > > (XEN) ****************************************
> > > > (XEN) Panic on CPU 0:
> > > > (XEN) Could not set up DOM0 guest OS
> > > > (XEN) ****************************************
> > > > (XEN)
> > > > (XEN) Reboot in five seconds...
> > > >
> > > > For Dom0 we took the linux-toradex kernel source, ran "make
xenconfig"
> > > > [6], which should add Xen-support and rebuilt the Image with "make
> > Image".
> > > > For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> > MEK-Module
> > > > [8] and only adjusted the "bootargs" parameter.
> > > > This Linux Image was able to run after we rebuilt and booted it
> > > > without
> > Xen.
> > >
> > > Which Device-Tree did you for boot Linux without Xen?
> >
> > Used the same Device-Tree Binary that we want to use with Xen which is
> > described in [7]. In [7] you can see rows 23 and 24, which include the
> > Devicetree as it is supplied by Toradex.
> > To boot linux with our dom0 DTB, the u-boot variable fdt_file was
> > changed from fsl-imx8qxp-colibri-eval-v3.dtb to
> > fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
>
>
> Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?

To boot linux without Xen the u-boot cmd 'boot' is used.
To boot Xen we use 'run xenmmcboot' (in an adjusted form to account for
different filenames and bootargs, see [4]).

>
> Regards,
> Peng.
>
> >
> > >
> > > >
> > > > Question 1:
> > > > The xen dokumentation always states, that the Dom0- and DomU-
> > > > Images have to be in zImage format, but according to [9] arm64
> > > > doesn't use zImage and [10] says the arm64/Image is equivalent to
zImage.
> > > > Can we assume that our arm64/Image should work, if everything else
> > > > was
> > > fine?
> > >
> > > This is a mistake in the documentation. Xen is able to load
arm64/Image.
> >
> > Thanks for the confirmation!
> >
> > >
> > > Can you give me a pointer to the documentation you are using? I will
> > update it.
> > >
> >
> > ' where "/root/image" is a Linux zImage.'
> > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extension
> > s#D
> > omU
> > _kernel_and_DTS
> >
> > > >
> > > > Question 2:
> > > > Our Xen was build with imx-xen repository [11] as source via NXP's
> > > > BSP according to [1] with MACHINE=imx8qxpmek.
> > > > Can we use the resulting Xen Hypervisor-Image for the Toradex
> > > > IMX8X Module, as it uses the same Processor (i.MX 8 QXP)?
> > >
> > > I am afraid this tree is not maintained by Xen Project but, IIRC,
> > > NXP. I
> > have CCed
> > > one person that may be able to help you.
> >
> > Thanks for connecting!
> >
> > > >
> > > > Question 3:
> > > > If the answers to Question 1 and Question 2 are yes, then what
> > > > else could we be missing?
> > > >
> > > >
> > > >
> > > > [1]
> > > >
> > https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-
> > > > imx-8x [2]
> > > > https://www.nxp.com/docs/en/user-
> > > guide/IMX_YOCTO_PROJECT_USERS_GUIDE.p
> > > > df [3]
> > > >
> > https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
> > > >
> > > > Bootscript:
> > > > [4] https://pastebin.com/dcj0K3Xb
> > > >
> > > > Complete Bootlog:
> > > > [5] https://pastebin.com/eLpZvd29
> > > >
> > > > Xenconfig according to Xen Wiki
> > > > [6]
> > > >
> > https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_l
> > > > atest_
> > > > Linux_kernel_support
> > > >
> > > > fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> > > > [7] https://pastebin.com/f4dGCYFf
> > > >
> > > > fsl-imx8qxp-mek-dom0.dts:
> > > > [8] https://pastebin.com/c3Ukc6mZ
> > > >
> > > > No arm64/zImage:
> > > > [9] https://community.nxp.com/thread/482861#comment-1051436
> > > >
> > > > arm64/Image is equivalent to zImage:
> > > > [10] http://markmail.org/message/advnw6lk5rcfyabp
> > > >
> > > > iMX-XEN Repo:
> > > > [11]
> > > >
> > https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35
> > > > _1.1.0
> > > >
> > > >
> > > >
> > > > Thank you very much for your reading this mail and helping with
> > > > our
> > issue!
> > > >
> > > > Best regards,
> > >
> > > Best regards,
> > >
> > > --
> > > Julien Grall
> >
> > Best regards,
> >
> > Daniel Wagner

RE: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

peng.fan at nxp

Aug 17, 2020, 3:01 AM

Post #6 of 15 (2011 views)

Permalink

> Subject: AW: Colibri imx8qxp: Missing kernel boot module
>
>
>
> > -----Urspr?ngliche Nachricht-----
> > Von: Peng Fan <peng.fan@nxp.com>
> > Gesendet: Montag, 17. August 2020 11:47
> > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; Julien Grall
> > <julien@xen.org>; xen-users@lists.xenproject.org
> > Betreff: RE: Colibri imx8qxp: Missing kernel boot module
> >
> > > Subject: AW: Colibri imx8qxp: Missing kernel boot module
> > >
> > >
> > >
> > > > -----Urspr?ngliche Nachricht-----
> > > > Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im Auftrag
> > > > von
> > > Julien
> > > > Grall
> > > > Gesendet: Freitag, 14. August 2020 21:46
> > > > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> > > > users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
> > > > Betreff: Re: Colibri imx8qxp: Missing kernel boot module
> > > >
> > > > On 10/08/2020 16:32, Daniel Wagner2 wrote:
> > > > > Hello xen-users,
> > > >
> > > > Hello,
> > > >
> > > > > we are trying to get Xen running on a Toradex "Colibri iMX8X"
> > > > > module (see [1] at the bottom of this email), which features the
> > > > > iMX8 QXP
> > > prozessor.
> > > > > We found that NXP has a Xen reference implementation [2][3] for
> > > > > their MEK Module and tried to port that to the Toradex module.
> > > > >
> > > > > When booting via the bootscript [4], which is unaltered except for
> > > > > the "dom0fdt_file" and "xenhyper_bootargs" variables, we get the
> > > > > following
> > > > > error:
> > > > >
> > > > > [...] (See [5] for complete bootlog)
> > > > > (XEN) *** LOADING DOMAIN 0 ***
> > > > > (XEN) Missing kernel boot module?
> > > > > (XEN)
> > > > > (XEN) ****************************************
> > > > > (XEN) Panic on CPU 0:
> > > > > (XEN) Could not set up DOM0 guest OS
> > > > > (XEN) ****************************************
> > > > > (XEN)
> > > > > (XEN) Reboot in five seconds...
> > > > >
> > > > > For Dom0 we took the linux-toradex kernel source, ran "make
> xenconfig"
> > > > > [6], which should add Xen-support and rebuilt the Image with "make
> > > Image".
> > > > > For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> > > MEK-Module
> > > > > [8] and only adjusted the "bootargs" parameter.
> > > > > This Linux Image was able to run after we rebuilt and booted it
> > > > > without
> > > Xen.
> > > >
> > > > Which Device-Tree did you for boot Linux without Xen?
> > >
> > > Used the same Device-Tree Binary that we want to use with Xen which is
> > > described in [7]. In [7] you can see rows 23 and 24, which include the
> > > Devicetree as it is supplied by Toradex.
> > > To boot linux with our dom0 DTB, the u-boot variable fdt_file was
> > > changed from fsl-imx8qxp-colibri-eval-v3.dtb to
> > > fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
> >
> >
> > Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
>
> To boot linux without Xen the u-boot cmd 'boot' is used.
> To boot Xen we use 'run xenmmcboot' (in an adjusted form to account for
> different filenames and bootargs, see [4]).

So kernel Image loaded to 0x80a00000, right?

Please load kernel image before run xenmmcboot if not.

Regards,
Peng.

>
> >
> > Regards,
> > Peng.
> >
> > >
> > > >
> > > > >
> > > > > Question 1:
> > > > > The xen dokumentation always states, that the Dom0- and DomU-
> > > > > Images have to be in zImage format, but according to [9] arm64
> > > > > doesn't use zImage and [10] says the arm64/Image is equivalent to
> zImage.
> > > > > Can we assume that our arm64/Image should work, if everything else
> > > > > was
> > > > fine?
> > > >
> > > > This is a mistake in the documentation. Xen is able to load
> arm64/Image.
> > >
> > > Thanks for the confirmation!
> > >
> > > >
> > > > Can you give me a pointer to the documentation you are using? I will
> > > update it.
> > > >
> > >
> > > ' where "/root/image" is a Linux zImage.'
> > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extension
> > > s#D
> > > omU
> > > _kernel_and_DTS
> > >
> > > > >
> > > > > Question 2:
> > > > > Our Xen was build with imx-xen repository [11] as source via NXP's
> > > > > BSP according to [1] with MACHINE=imx8qxpmek.
> > > > > Can we use the resulting Xen Hypervisor-Image for the Toradex
> > > > > IMX8X Module, as it uses the same Processor (i.MX 8 QXP)?
> > > >
> > > > I am afraid this tree is not maintained by Xen Project but, IIRC,
> > > > NXP. I
> > > have CCed
> > > > one person that may be able to help you.
> > >
> > > Thanks for connecting!
> > >
> > > > >
> > > > > Question 3:
> > > > > If the answers to Question 1 and Question 2 are yes, then what
> > > > > else could we be missing?
> > > > >
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > >
> https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-
> > > > > imx-8x [2]
> > > > > https://www.nxp.com/docs/en/user-
> > > > guide/IMX_YOCTO_PROJECT_USERS_GUIDE.p
> > > > > df [3]
> > > > >
> > >
> https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
> > > > >
> > > > > Bootscript:
> > > > > [4] https://pastebin.com/dcj0K3Xb
> > > > >
> > > > > Complete Bootlog:
> > > > > [5] https://pastebin.com/eLpZvd29
> > > > >
> > > > > Xenconfig according to Xen Wiki
> > > > > [6]
> > > > >
> > >
> https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_l
> > > > > atest_
> > > > > Linux_kernel_support
> > > > >
> > > > > fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> > > > > [7] https://pastebin.com/f4dGCYFf
> > > > >
> > > > > fsl-imx8qxp-mek-dom0.dts:
> > > > > [8] https://pastebin.com/c3Ukc6mZ
> > > > >
> > > > > No arm64/zImage:
> > > > > [9] https://community.nxp.com/thread/482861#comment-1051436
> > > > >
> > > > > arm64/Image is equivalent to zImage:
> > > > > [10] http://markmail.org/message/advnw6lk5rcfyabp
> > > > >
> > > > > iMX-XEN Repo:
> > > > > [11]
> > > > >
> > >
> https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35
> > > > > _1.1.0
> > > > >
> > > > >
> > > > >
> > > > > Thank you very much for your reading this mail and helping with
> > > > > our
> > > issue!
> > > > >
> > > > > Best regards,
> > > >
> > > > Best regards,
> > > >
> > > > --
> > > > Julien Grall
> > >
> > > Best regards,
> > >
> > > Daniel Wagner

AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Aug 17, 2020, 3:29 AM

Post #7 of 15 (2011 views)

Permalink

> -----Urspr?ngliche Nachricht-----
> Von: Peng Fan <peng.fan@nxp.com>
> Gesendet: Montag, 17. August 2020 12:02
> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> users@lists.xenproject.org
> Betreff: RE: Colibri imx8qxp: Missing kernel boot module
>
> > Subject: AW: Colibri imx8qxp: Missing kernel boot module
> >
> >
> >
> > > -----Urspr?ngliche Nachricht-----
> > > Von: Peng Fan <peng.fan@nxp.com>
> > > Gesendet: Montag, 17. August 2020 11:47
> > > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; Julien Grall
> > > <julien@xen.org>; xen-users@lists.xenproject.org
> > > Betreff: RE: Colibri imx8qxp: Missing kernel boot module
> > >
> > > > Subject: AW: Colibri imx8qxp: Missing kernel boot module
> > > >
> > > >
> > > >
> > > > > -----Urspr?ngliche Nachricht-----
> > > > > Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im
> > > > > Auftrag von
> > > > Julien
> > > > > Grall
> > > > > Gesendet: Freitag, 14. August 2020 21:46
> > > > > An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
> > > > > users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
> > > > > Betreff: Re: Colibri imx8qxp: Missing kernel boot module
> > > > >
> > > > > On 10/08/2020 16:32, Daniel Wagner2 wrote:
> > > > > > Hello xen-users,
> > > > >
> > > > > Hello,
> > > > >
> > > > > > we are trying to get Xen running on a Toradex "Colibri iMX8X"
> > > > > > module (see [1] at the bottom of this email), which features
> > > > > > the
> > > > > > iMX8 QXP
> > > > prozessor.
> > > > > > We found that NXP has a Xen reference implementation [2][3]
> > > > > > for their MEK Module and tried to port that to the Toradex
module.
> > > > > >
> > > > > > When booting via the bootscript [4], which is unaltered except
> > > > > > for the "dom0fdt_file" and "xenhyper_bootargs" variables, we
> > > > > > get the following
> > > > > > error:
> > > > > >
> > > > > > [...] (See [5] for complete bootlog)
> > > > > > (XEN) *** LOADING DOMAIN 0 ***
> > > > > > (XEN) Missing kernel boot module?
> > > > > > (XEN)
> > > > > > (XEN) ****************************************
> > > > > > (XEN) Panic on CPU 0:
> > > > > > (XEN) Could not set up DOM0 guest OS
> > > > > > (XEN) ****************************************
> > > > > > (XEN)
> > > > > > (XEN) Reboot in five seconds...
> > > > > >
> > > > > > For Dom0 we took the linux-toradex kernel source, ran "make
> > xenconfig"
> > > > > > [6], which should add Xen-support and rebuilt the Image with
> > > > > > "make
> > > > Image".
> > > > > > For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> > > > MEK-Module
> > > > > > [8] and only adjusted the "bootargs" parameter.
> > > > > > This Linux Image was able to run after we rebuilt and booted
> > > > > > it without
> > > > Xen.
> > > > >
> > > > > Which Device-Tree did you for boot Linux without Xen?
> > > >
> > > > Used the same Device-Tree Binary that we want to use with Xen
> > > > which is described in [7]. In [7] you can see rows 23 and 24,
> > > > which include the Devicetree as it is supplied by Toradex.
> > > > To boot linux with our dom0 DTB, the u-boot variable fdt_file was
> > > > changed from fsl-imx8qxp-colibri-eval-v3.dtb to
> > > > fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
> > >
> > >
> > > Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
> >
> > To boot linux without Xen the u-boot cmd 'boot' is used.
> > To boot Xen we use 'run xenmmcboot' (in an adjusted form to account
> > for different filenames and bootargs, see [4]).
>
> So kernel Image loaded to 0x80a00000, right?
>
> Please load kernel image before run xenmmcboot if not.

The xenmmcboot script [4] line 25 loads the linux image.
Inserting the values for the variables in this line gives:
'fatload mmc 0:1 0x92000000 image'
So we load the kernel image to 0x92000000. Is this not the right adress?
How can the right adress, to load the kernel image to, be determined?
Loading it to 0x80a00000 yields the same error ("Missing kernel boot
module?")

Regards,
Daniel

> Regards,
> Peng.
>
> >
> > >
> > > Regards,
> > > Peng.
> > >
> > > >
> > > > >
> > > > > >
> > > > > > Question 1:
> > > > > > The xen dokumentation always states, that the Dom0- and DomU-
> > > > > > Images have to be in zImage format, but according to [9] arm64
> > > > > > doesn't use zImage and [10] says the arm64/Image is equivalent
> > > > > > to
> > zImage.
> > > > > > Can we assume that our arm64/Image should work, if everything
> > > > > > else was
> > > > > fine?
> > > > >
> > > > > This is a mistake in the documentation. Xen is able to load
> > arm64/Image.
> > > >
> > > > Thanks for the confirmation!
> > > >
> > > > >
> > > > > Can you give me a pointer to the documentation you are using? I
> > > > > will
> > > > update it.
> > > > >
> > > >
> > > > ' where "/root/image" is a Linux zImage.'
> > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Exten
> > > > sion
> > > > s#D
> > > > omU
> > > > _kernel_and_DTS
> > > >
> > > > > >
> > > > > > Question 2:
> > > > > > Our Xen was build with imx-xen repository [11] as source via
> > > > > > NXP's BSP according to [1] with MACHINE=imx8qxpmek.
> > > > > > Can we use the resulting Xen Hypervisor-Image for the Toradex
> > > > > > IMX8X Module, as it uses the same Processor (i.MX 8 QXP)?
> > > > >
> > > > > I am afraid this tree is not maintained by Xen Project but,
> > > > > IIRC, NXP. I
> > > > have CCed
> > > > > one person that may be able to help you.
> > > >
> > > > Thanks for connecting!
> > > >
> > > > > >
> > > > > > Question 3:
> > > > > > If the answers to Question 1 and Question 2 are yes, then what
> > > > > > else could we be missing?
> > > > > >
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > >
> > https://www.toradex.com/de/computer-on-modules/colibri-arm-family/nxp-
> > > > > > imx-8x [2]
> > > > > > https://www.nxp.com/docs/en/user-
> > > > > guide/IMX_YOCTO_PROJECT_USERS_GUIDE.p
> > > > > > df [3]
> > > > > >
> > > >
> > https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf
> > > > > >
> > > > > > Bootscript:
> > > > > > [4] https://pastebin.com/dcj0K3Xb
> > > > > >
> > > > > > Complete Bootlog:
> > > > > > [5] https://pastebin.com/eLpZvd29
> > > > > >
> > > > > > Xenconfig according to Xen Wiki [6]
> > > > > >
> > > >
> > https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source#Compiling_l
> > > > > > atest_
> > > > > > Linux_kernel_support
> > > > > >
> > > > > > fsl-imx8qxp-colibri-eval-v3-dom0.dts:
> > > > > > [7] https://pastebin.com/f4dGCYFf
> > > > > >
> > > > > > fsl-imx8qxp-mek-dom0.dts:
> > > > > > [8] https://pastebin.com/c3Ukc6mZ
> > > > > >
> > > > > > No arm64/zImage:
> > > > > > [9] https://community.nxp.com/thread/482861#comment-1051436
> > > > > >
> > > > > > arm64/Image is equivalent to zImage:
> > > > > > [10] http://markmail.org/message/advnw6lk5rcfyabp
> > > > > >
> > > > > > iMX-XEN Repo:
> > > > > > [11]
> > > > > >
> > > >
> > https://source.codeaurora.org/external/imx/imx-xen/tree/?h=imx_4.19.35
> > > > > > _1.1.0
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thank you very much for your reading this mail and helping
> > > > > > with our
> > > > issue!
> > > > > >
> > > > > > Best regards,
> > > > >
> > > > > Best regards,
> > > > >
> > > > > --
> > > > > Julien Grall
> > > >
> > > > Best regards,
> > > >
> > > > Daniel Wagner

Re: AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

julien at xen

Aug 20, 2020, 2:42 AM

Post #8 of 15 (2005 views)

Permalink

Hi Daniel,

On 17/08/2020 11:29, Daniel Wagner2 wrote:
>
>
>> -----Urspr?ngliche Nachricht-----
>> Von: Peng Fan <peng.fan@nxp.com>
>> Gesendet: Montag, 17. August 2020 12:02
>> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
>> users@lists.xenproject.org
>> Betreff: RE: Colibri imx8qxp: Missing kernel boot module
>>
>>> Subject: AW: Colibri imx8qxp: Missing kernel boot module
>>>
>>>
>>>
>>>> -----Urspr?ngliche Nachricht-----
>>>> Von: Peng Fan <peng.fan@nxp.com>
>>>> Gesendet: Montag, 17. August 2020 11:47
>>>> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; Julien Grall
>>>> <julien@xen.org>; xen-users@lists.xenproject.org
>>>> Betreff: RE: Colibri imx8qxp: Missing kernel boot module
>>>>
>>>>> Subject: AW: Colibri imx8qxp: Missing kernel boot module
>>>>>
>>>>>
>>>>>
>>>>>> -----Urspr?ngliche Nachricht-----
>>>>>> Von: Xen-users <xen-users-bounces@lists.xenproject.org> Im
>>>>>> Auftrag von
>>>>> Julien
>>>>>> Grall
>>>>>> Gesendet: Freitag, 14. August 2020 21:46
>>>>>> An: Daniel Wagner2 <Daniel.Wagner2@itk-engineering.de>; xen-
>>>>>> users@lists.xenproject.org; Peng Fan <peng.fan@nxp.com>
>>>>>> Betreff: Re: Colibri imx8qxp: Missing kernel boot module
>>>>>>
>>>>>> On 10/08/2020 16:32, Daniel Wagner2 wrote:
>>>>>>> Hello xen-users,
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>> we are trying to get Xen running on a Toradex "Colibri iMX8X"
>>>>>>> module (see [1] at the bottom of this email), which features
>>>>>>> the
>>>>>>> iMX8 QXP
>>>>> prozessor.
>>>>>>> We found that NXP has a Xen reference implementation [2][3]
>>>>>>> for their MEK Module and tried to port that to the Toradex
> module.
>>>>>>>
>>>>>>> When booting via the bootscript [4], which is unaltered except
>>>>>>> for the "dom0fdt_file" and "xenhyper_bootargs" variables, we
>>>>>>> get the following
>>>>>>> error:
>>>>>>>
>>>>>>> [...] (See [5] for complete bootlog)
>>>>>>> (XEN) *** LOADING DOMAIN 0 ***
>>>>>>> (XEN) Missing kernel boot module?
>>>>>>> (XEN)
>>>>>>> (XEN) ****************************************
>>>>>>> (XEN) Panic on CPU 0:
>>>>>>> (XEN) Could not set up DOM0 guest OS
>>>>>>> (XEN) ****************************************
>>>>>>> (XEN)
>>>>>>> (XEN) Reboot in five seconds...
>>>>>>>
>>>>>>> For Dom0 we took the linux-toradex kernel source, ran "make
>>> xenconfig"
>>>>>>> [6], which should add Xen-support and rebuilt the Image with
>>>>>>> "make
>>>>> Image".
>>>>>>> For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
>>>>> MEK-Module
>>>>>>> [8] and only adjusted the "bootargs" parameter.
>>>>>>> This Linux Image was able to run after we rebuilt and booted
>>>>>>> it without
>>>>> Xen.
>>>>>>
>>>>>> Which Device-Tree did you for boot Linux without Xen?
>>>>>
>>>>> Used the same Device-Tree Binary that we want to use with Xen
>>>>> which is described in [7]. In [7] you can see rows 23 and 24,
>>>>> which include the Devicetree as it is supplied by Toradex.
>>>>> To boot linux with our dom0 DTB, the u-boot variable fdt_file was
>>>>> changed from fsl-imx8qxp-colibri-eval-v3.dtb to
>>>>> fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
>>>>
>>>>
>>>> Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
>>>
>>> To boot linux without Xen the u-boot cmd 'boot' is used.
>>> To boot Xen we use 'run xenmmcboot' (in an adjusted form to account
>>> for different filenames and bootargs, see [4]).
>>
>> So kernel Image loaded to 0x80a00000, right?
>>
>> Please load kernel image before run xenmmcboot if not.
>
> The xenmmcboot script [4] line 25 loads the linux image.
> Inserting the values for the variables in this line gives:
> 'fatload mmc 0:1 0x92000000 image'
> So we load the kernel image to 0x92000000. Is this not the right adress?
> How can the right adress, to load the kernel image to, be determined?

With U-boot, you will have to unfortunately determine the address
manually. I usually use the address provided by U-boot for baremetal
boot and load both Xen and Linux close to each other.

> Loading it to 0x80a00000 yields the same error ("Missing kernel boot
> module?")

It may be possible that the Device-Tree node is incorrect. Would it be
possible to dump the Device-Tree node /chosen/ in either U-boot or Xen?

Cheers,

--
Julien Grall

RE: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Aug 22, 2020, 5:00 AM

Post #9 of 15 (2000 views)

Permalink

Hello Julien,

> >>>>>> On 10/08/2020 16:32, Daniel Wagner2 wrote:
> >>>>>>> Hello xen-users,
> >>>>>>> we are trying to get Xen running on a Toradex "Colibri iMX8X"
> >>>>>>> module (see [1] at the bottom of this email), which features the
> >>>>>>> iMX8 QXP
> >>>>> prozessor.
> >>>>>>> We found that NXP has a Xen reference implementation [2][3] for
> >>>>>>> their MEK Module and tried to port that to the Toradex
> > module.
> >>>>>>>
> >>>>>>> When booting via the bootscript [4], which is unaltered except
> >>>>>>> for the "dom0fdt_file" and "xenhyper_bootargs" variables, we get
> >>>>>>> the following
> >>>>>>> error:
> >>>>>>>
> >>>>>>> [...] (See [5] for complete bootlog)
> >>>>>>> (XEN) *** LOADING DOMAIN 0 ***
> >>>>>>> (XEN) Missing kernel boot module?
> >>>>>>> (XEN)
> >>>>>>> (XEN) ****************************************
> >>>>>>> (XEN) Panic on CPU 0:
> >>>>>>> (XEN) Could not set up DOM0 guest OS
> >>>>>>> (XEN) ****************************************
> >>>>>>> (XEN)
> >>>>>>> (XEN) Reboot in five seconds...
> >>>>>>>
> >>>>>>> For Dom0 we took the linux-toradex kernel source, ran "make
> >>> xenconfig"
> >>>>>>> [6], which should add Xen-support and rebuilt the Image with
> >>>>>>> "make
> >>>>> Image".
> >>>>>>> For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> >>>>> MEK-Module
> >>>>>>> [8] and only adjusted the "bootargs" parameter.
> >>>>>>> This Linux Image was able to run after we rebuilt and booted it
> >>>>>>> without
> >>>>> Xen.
> >>>>>>
> >>>>>> Which Device-Tree did you for boot Linux without Xen?
> >>>>>
> >>>>> Used the same Device-Tree Binary that we want to use with Xen
> >>>>> which is described in [7]. In [7] you can see rows 23 and 24,
> >>>>> which include the Devicetree as it is supplied by Toradex.
> >>>>> To boot linux with our dom0 DTB, the u-boot variable fdt_file was
> >>>>> changed from fsl-imx8qxp-colibri-eval-v3.dtb to
> >>>>> fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
> >>>>
> >>>>
> >>>> Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
> >>>
> >>> To boot linux without Xen the u-boot cmd 'boot' is used.
> >>> To boot Xen we use 'run xenmmcboot' (in an adjusted form to account
> >>> for different filenames and bootargs, see [4]).
> >>
> >> So kernel Image loaded to 0x80a00000, right?
> >>
> >> Please load kernel image before run xenmmcboot if not.
> >
> > The xenmmcboot script [4] line 25 loads the linux image.
> > Inserting the values for the variables in this line gives:
> > 'fatload mmc 0:1 0x92000000 image'
> > So we load the kernel image to 0x92000000. Is this not the right adress?
> > How can the right adress, to load the kernel image to, be determined?
>
> With U-boot, you will have to unfortunately determine the address
manually. I
> usually use the address provided by U-boot for baremetal boot and load
both
> Xen and Linux close to each other.
>
> > Loading it to 0x80a00000 yields the same error ("Missing kernel boot
> > module?")
>
> It may be possible that the Device-Tree node is incorrect. Would it be
possible to
> dump the Device-Tree node /chosen/ in either U-boot or Xen?
>

We tried to load xen, dom0 und fdt close together in an effort to replicate
your approach.
Thats why we now load
xen @ 0x8200 0000
dom0 @ 0x8210 0000
fdt @ 0x8370 0000
To account for these changes, the /chosen node was also adjusted.
/chosen (for readibility as pastebin):
https://pastebin.com/v6pmm1iq

chosen {
bootargs = "console=ttyLP3,115200
earlycon=lpuart32,0x5a090000,115200";
stdout-path = "/serial@5a090000";
#address-cells = <0x00000002>;
#size-cells = <0x00000002>;
module@0 {
bootargs = "earlycon=xen console=hvc0 loglevel=8
root=/dev/mmcblk0p2 rootwait rw";
xen,dom0-bootargs = "console=dtuart dtuart=ttyLP3
earlycon=lpuart3,0x5a090000,115200";
compatible = "xen,linux-zimage", "xen,multiboot-module";
reg = <0x00000000 0x82100000 0x00000000 0x01490a00>;
};
};

Regards,

Daniel

Re: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

julien at xen

Aug 25, 2020, 9:49 AM

Post #10 of 15 (1995 views)

Permalink

Hi Daniel,

On 22/08/2020 13:00, Daniel Wagner2 wrote:
>>>>>>>> On 10/08/2020 16:32, Daniel Wagner2 wrote:
>>>>>>>>> Hello xen-users,
>>>>>>>>> we are trying to get Xen running on a Toradex "Colibri iMX8X"
>>>>>>>>> module (see [1] at the bottom of this email), which features the
>>>>>>>>> iMX8 QXP
>>>>>>> prozessor.
>>>>>>>>> We found that NXP has a Xen reference implementation [2][3] for
>>>>>>>>> their MEK Module and tried to port that to the Toradex
>>> module.
>>>>>>>>>
>>>>>>>>> When booting via the bootscript [4], which is unaltered except
>>>>>>>>> for the "dom0fdt_file" and "xenhyper_bootargs" variables, we get
>>>>>>>>> the following
>>>>>>>>> error:
>>>>>>>>>
>>>>>>>>> [...] (See [5] for complete bootlog)
>>>>>>>>> (XEN) *** LOADING DOMAIN 0 ***
>>>>>>>>> (XEN) Missing kernel boot module?
>>>>>>>>> (XEN)
>>>>>>>>> (XEN) ****************************************
>>>>>>>>> (XEN) Panic on CPU 0:
>>>>>>>>> (XEN) Could not set up DOM0 guest OS
>>>>>>>>> (XEN) ****************************************
>>>>>>>>> (XEN)
>>>>>>>>> (XEN) Reboot in five seconds...
>>>>>>>>>
>>>>>>>>> For Dom0 we took the linux-toradex kernel source, ran "make
>>>>> xenconfig"
>>>>>>>>> [6], which should add Xen-support and rebuilt the Image with
>>>>>>>>> "make
>>>>>>> Image".
>>>>>>>>> For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
>>>>>>> MEK-Module
>>>>>>>>> [8] and only adjusted the "bootargs" parameter.
>>>>>>>>> This Linux Image was able to run after we rebuilt and booted it
>>>>>>>>> without
>>>>>>> Xen.
>>>>>>>>
>>>>>>>> Which Device-Tree did you for boot Linux without Xen?
>>>>>>>
>>>>>>> Used the same Device-Tree Binary that we want to use with Xen
>>>>>>> which is described in [7]. In [7] you can see rows 23 and 24,
>>>>>>> which include the Devicetree as it is supplied by Toradex.
>>>>>>> To boot linux with our dom0 DTB, the u-boot variable fdt_file was
>>>>>>> changed from fsl-imx8qxp-colibri-eval-v3.dtb to
>>>>>>> fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
>>>>>>
>>>>>>
>>>>>> Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
>>>>>
>>>>> To boot linux without Xen the u-boot cmd 'boot' is used.
>>>>> To boot Xen we use 'run xenmmcboot' (in an adjusted form to account
>>>>> for different filenames and bootargs, see [4]).
>>>>
>>>> So kernel Image loaded to 0x80a00000, right?
>>>>
>>>> Please load kernel image before run xenmmcboot if not.
>>>
>>> The xenmmcboot script [4] line 25 loads the linux image.
>>> Inserting the values for the variables in this line gives:
>>> 'fatload mmc 0:1 0x92000000 image'
>>> So we load the kernel image to 0x92000000. Is this not the right adress?
>>> How can the right adress, to load the kernel image to, be determined?
>>
>> With U-boot, you will have to unfortunately determine the address
> manually. I
>> usually use the address provided by U-boot for baremetal boot and load
> both
>> Xen and Linux close to each other.
>>
>>> Loading it to 0x80a00000 yields the same error ("Missing kernel boot
>>> module?")
>>
>> It may be possible that the Device-Tree node is incorrect. Would it be
> possible to
>> dump the Device-Tree node /chosen/ in either U-boot or Xen?
>>
>
> We tried to load xen, dom0 und fdt close together in an effort to replicate
> your approach.
> Thats why we now load
> xen @ 0x8200 0000
> dom0 @ 0x8210 0000
> fdt @ 0x8370 0000
> To account for these changes, the /chosen node was also adjusted.
> /chosen (for readibility as pastebin):
> https://pastebin.com/v6pmm1iq
>
> chosen {
> bootargs = "console=ttyLP3,115200
> earlycon=lpuart32,0x5a090000,115200";
> stdout-path = "/serial@5a090000";
> #address-cells = <0x00000002>;
> #size-cells = <0x00000002>;
> module@0 {
> bootargs = "earlycon=xen console=hvc0 loglevel=8
> root=/dev/mmcblk0p2 rootwait rw";
> xen,dom0-bootargs = "console=dtuart dtuart=ttyLP3
> earlycon=lpuart3,0x5a090000,115200";
> compatible = "xen,linux-zimage", "xen,multiboot-module";
> reg = <0x00000000 0x82100000 0x00000000 0x01490a00>;
> };
> };

Thank you for the input. At first glance, I don't see the issue with the
node but it looks like Xen is not happy with it.

I am not entirely sure why yet. Do you mind to enable earlyprintk for
your platform? With that you should be able to get more information
about the list of modules discovered.

Cheers,

--
Julien Grall

AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Sep 1, 2020, 7:16 AM

Post #11 of 15 (1991 views)

Permalink

Hello,

> On 22/08/2020 13:00, Daniel Wagner2 wrote:
> >>>>>>>> On 10/08/2020 16:32, Daniel Wagner2 wrote:
> >>>>>>>>> Hello xen-users,
> >>>>>>>>> we are trying to get Xen running on a Toradex "Colibri iMX8X"
> >>>>>>>>> module (see [1] at the bottom of this email), which features
> >>>>>>>>> the
> >>>>>>>>> iMX8 QXP
> >>>>>>> prozessor.
> >>>>>>>>> We found that NXP has a Xen reference implementation [2][3]
> >>>>>>>>> for their MEK Module and tried to port that to the Toradex
> >>> module.
> >>>>>>>>>
> >>>>>>>>> When booting via the bootscript [4], which is unaltered except
> >>>>>>>>> for the "dom0fdt_file" and "xenhyper_bootargs" variables, we
> >>>>>>>>> get the following
> >>>>>>>>> error:
> >>>>>>>>>
> >>>>>>>>> [...] (See [5] for complete bootlog)
> >>>>>>>>> (XEN) *** LOADING DOMAIN 0 ***
> >>>>>>>>> (XEN) Missing kernel boot module?
> >>>>>>>>> (XEN)
> >>>>>>>>> (XEN) ****************************************
> >>>>>>>>> (XEN) Panic on CPU 0:
> >>>>>>>>> (XEN) Could not set up DOM0 guest OS
> >>>>>>>>> (XEN) ****************************************
> >>>>>>>>> (XEN)
> >>>>>>>>> (XEN) Reboot in five seconds...
> >>>>>>>>>
> >>>>>>>>> For Dom0 we took the linux-toradex kernel source, ran "make
> >>>>> xenconfig"
> >>>>>>>>> [6], which should add Xen-support and rebuilt the Image with
> >>>>>>>>> "make
> >>>>>>> Image".
> >>>>>>>>> For dom0 DTB [7] we copied the dom0 DTS NXP uses for their
> >>>>>>> MEK-Module
> >>>>>>>>> [8] and only adjusted the "bootargs" parameter.
> >>>>>>>>> This Linux Image was able to run after we rebuilt and booted
> >>>>>>>>> it without
> >>>>>>> Xen.
> >>>>>>>>
> >>>>>>>> Which Device-Tree did you for boot Linux without Xen?
> >>>>>>>
> >>>>>>> Used the same Device-Tree Binary that we want to use with Xen
> >>>>>>> which is described in [7]. In [7] you can see rows 23 and 24,
> >>>>>>> which include the Devicetree as it is supplied by Toradex.
> >>>>>>> To boot linux with our dom0 DTB, the u-boot variable fdt_file
> >>>>>>> was changed from fsl-imx8qxp-colibri-eval-v3.dtb to
> >>>>>>> fsl-imx8qxp-colibri-eval-v3-dom0.dtb.
> >>>>>>
> >>>>>>
> >>>>>> Did you use the U-Boot cmd 'xenmmcboot' or 'xennetboot'?
> >>>>>
> >>>>> To boot linux without Xen the u-boot cmd 'boot' is used.
> >>>>> To boot Xen we use 'run xenmmcboot' (in an adjusted form to
> >>>>> account for different filenames and bootargs, see [4]).
> >>>>
> >>>> So kernel Image loaded to 0x80a00000, right?
> >>>>
> >>>> Please load kernel image before run xenmmcboot if not.
> >>>
> >>> The xenmmcboot script [4] line 25 loads the linux image.
> >>> Inserting the values for the variables in this line gives:
> >>> 'fatload mmc 0:1 0x92000000 image'
> >>> So we load the kernel image to 0x92000000. Is this not the right
adress?
> >>> How can the right adress, to load the kernel image to, be determined?
> >>
> >> With U-boot, you will have to unfortunately determine the address
> > manually. I
> >> usually use the address provided by U-boot for baremetal boot and
> >> load
> > both
> >> Xen and Linux close to each other.
> >>
> >>> Loading it to 0x80a00000 yields the same error ("Missing kernel boot
> >>> module?")
> >>
> >> It may be possible that the Device-Tree node is incorrect. Would it
> >> be
> > possible to
> >> dump the Device-Tree node /chosen/ in either U-boot or Xen?
> >>
> >
> > We tried to load xen, dom0 und fdt close together in an effort to
> > replicate your approach.
> > Thats why we now load
> > xen @ 0x8200 0000
> > dom0 @ 0x8210 0000
> > fdt @ 0x8370 0000
> > To account for these changes, the /chosen node was also adjusted.
> > /chosen (for readibility as pastebin):
> > https://pastebin.com/v6pmm1iq
> >
> > chosen {
> > bootargs = "console=ttyLP3,115200
> > earlycon=lpuart32,0x5a090000,115200";
> > stdout-path = "/serial@5a090000";
> > #address-cells = <0x00000002>;
> > #size-cells = <0x00000002>;
> > module@0 {
> > bootargs = "earlycon=xen console=hvc0 loglevel=8
> > root=/dev/mmcblk0p2 rootwait rw";
> > xen,dom0-bootargs = "console=dtuart dtuart=ttyLP3
> > earlycon=lpuart3,0x5a090000,115200";
> > compatible = "xen,linux-zimage", "xen,multiboot-module";
> > reg = <0x00000000 0x82100000 0x00000000 0x01490a00>;
> > };
> > };
>
> Thank you for the input. At first glance, I don't see the issue with the
node
> but it looks like Xen is not happy with it.
>
> I am not entirely sure why yet. Do you mind to enable earlyprintk for your
> platform? With that you should be able to get more information about the
> list of modules discovered.

I didn't manage to enable earlyprintk.
But I enabled kconfig "Developer Checks": "Verbose debug messages" and
"Devicetree debug messages" and put some extra printks in the Xen Source and
was able to locate the problem.

When parsing the fdt, the memory@80000000 node throws an error, which stops
the parsing before the /chosen node was found and so no kernel boot module
was found for dom0.

I bypassed this by putting the /chosen node at the top of the fdt, so that
module@0 gets parsed before the functions arrives at the memory node. After
this change, Dom0 was successfully booted by Xen.

I have created a pastebin https://pastebin.com/JBjKNvPP for future
reference.
At pastebin line 215 Xen already found the kernel boot module.
So I put
"boot_fdt_info(device_tree_flattened, fdt_paddr);"
in start_xen (arch/arm/setup.c) after the
"console_init_postirq();"
call, so tha the function is run a second time, but this time the outputs
are shown in the bootlog (starting pastebin l. 247).
Starting at pastebin line 337, parser arrives at memory node.
Line 349 is what is specified in the fdt (which I copy here for reference)

memory@80000000 {
device_type = "memory";
reg = <0x00000000 0x80000000 0 0x40000000>;
};

The first bank found (l. 349) is the one from the fdt node. I am not sure
where the second bank (l. 350) comes from. The the second bank's size=0 ist
what causes the parse to fail and will Xen prevent from finding the boot
kernel for dom0 if the /chosen node comes after the memory node in the fdt.

Thanks for the help wih this issue Julien and Peng!

Regards,

Daniel

Re: AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

julien at xen

Sep 4, 2020, 10:19 AM

Post #12 of 15 (1990 views)

Permalink

On 01/09/2020 15:16, Daniel Wagner2 wrote:
> Hello,

Hi Daniel,

>> On 22/08/2020 13:00, Daniel Wagner2 wrote:
>>>>>>>>>> On 10/08/2020 16:32, Daniel Wagner2 wrote:
>> I am not entirely sure why yet. Do you mind to enable earlyprintk for your
>> platform? With that you should be able to get more information about the
>> list of modules discovered.
>
> I didn't manage to enable earlyprintk.
> But I enabled kconfig "Developer Checks": "Verbose debug messages" and
> "Devicetree debug messages" and put some extra printks in the Xen Source and
> was able to locate the problem.
>
> When parsing the fdt, the memory@80000000 node throws an error, which stops
> the parsing before the /chosen node was found and so no kernel boot module
> was found for dom0.
>
> I bypassed this by putting the /chosen node at the top of the fdt, so that
> module@0 gets parsed before the functions arrives at the memory node. After
> this change, Dom0 was successfully booted by Xen.
>
> I have created a pastebin https://pastebin.com/JBjKNvPP for future
> reference.
> At pastebin line 215 Xen already found the kernel boot module.
> So I put
> "boot_fdt_info(device_tree_flattened, fdt_paddr);"
> in start_xen (arch/arm/setup.c) after the
> "console_init_postirq();"
> call, so tha the function is run a second time, but this time the outputs
> are shown in the bootlog (starting pastebin l. 247).
> Starting at pastebin line 337, parser arrives at memory node.
> Line 349 is what is specified in the fdt (which I copy here for reference)
>
> memory@80000000 {
> device_type = "memory";
> reg = <0x00000000 0x80000000 0 0x40000000>;
> };
>
> The first bank found (l. 349) is the one from the fdt node. I am not sure
> where the second bank (l. 350) comes from. The the second bank's size=0 ist
> what causes the parse to fail and will Xen prevent from finding the boot
> kernel for dom0 if the /chosen node comes after the memory node in the fdt.

Thank you for digging through the problem. I find a bit odd that Xen
discover a second bank. It might be possible we don't parse the
Device-Tree correctly.

I wasn't able to find the node in the device-tree you provided in the
initial e-mail. Would you mind to post the full device-tree?

Cheers,

--
Julien Grall

AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Sep 9, 2020, 4:20 AM

Post #13 of 15 (1981 views)

Permalink

> >> I am not entirely sure why yet. Do you mind to enable earlyprintk for
> >> your platform? With that you should be able to get more information
> >> about the list of modules discovered.
> >
> > I didn't manage to enable earlyprintk.
> > But I enabled kconfig "Developer Checks": "Verbose debug messages" and
> > "Devicetree debug messages" and put some extra printks in the Xen
> > Source and was able to locate the problem.
> >
> > When parsing the fdt, the memory@80000000 node throws an error, which
> > stops the parsing before the /chosen node was found and so no kernel
> > boot module was found for dom0.
> >
> > I bypassed this by putting the /chosen node at the top of the fdt, so
> > that
> > module@0 gets parsed before the functions arrives at the memory node.
> > After this change, Dom0 was successfully booted by Xen.
> >
> > I have created a pastebin https://pastebin.com/JBjKNvPP for future
> > reference.
> > At pastebin line 215 Xen already found the kernel boot module.
> > So I put
> > "boot_fdt_info(device_tree_flattened, fdt_paddr);"
> > in start_xen (arch/arm/setup.c) after the "console_init_postirq();"
> > call, so tha the function is run a second time, but this time the
> > outputs are shown in the bootlog (starting pastebin l. 247).
> > Starting at pastebin line 337, parser arrives at memory node.
> > Line 349 is what is specified in the fdt (which I copy here for
> > reference)
> >
> > memory@80000000 {
> > device_type = "memory";
> > reg = <0x00000000 0x80000000 0 0x40000000>; };
> >
> > The first bank found (l. 349) is the one from the fdt node. I am not
> > sure where the second bank (l. 350) comes from. The the second bank's
> > size=0 ist what causes the parse to fail and will Xen prevent from
> > finding the boot kernel for dom0 if the /chosen node comes after the
memory
> node in the fdt.
>
> Thank you for digging through the problem. I find a bit odd that Xen
discover a
> second bank. It might be possible we don't parse the Device-Tree
correctly.
>
> I wasn't able to find the node in the device-tree you provided in the
initial e-mail.
> Would you mind to post the full device-tree?

Hello Julien,

this is the full version of the fdt that threw the error:
https://pastebin.com/63TZ9z3k
The problematic memory node appears in line 126

Greetings
Daniel

Re: AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

julien at xen

Sep 11, 2020, 6:22 AM

Post #14 of 15 (1980 views)

Permalink

On 09/09/2020 12:20, Daniel Wagner2 wrote:
> Hello Julien,
Hi Daniel,

>
> this is the full version of the fdt that threw the error:
> https://pastebin.com/63TZ9z3k
> The problematic memory node appears in line 126

Thanks! The output looks corrupted as some of the lines are not valid DTB:

fsl,pins = * 0x000000009300184c [0x00000048];

Although, I am not sure if it is just U-boot dumping the DTB differently.

Anyway, after removing the "corrupted" line, I managed to get a compile
the DTB. I don't have a Colibri IMX8QXP. However, given this is an early
parsing error, I have just embed the DTB in Xen binary via CONFIG_DTB_FILE.

Unfortunately I couldn't reproduce your error. This either suggests the
DTB gets corrupted or Xen doesn't access the DTB with the correct memory
attribute.

Do you have the DTB in hand?

Cheers,

--
Julien Grall

AW: AW: Colibri imx8qxp: Missing kernel boot module [ In reply to ]

Daniel.Wagner2 at itk-engineering

Sep 14, 2020, 7:26 AM

Post #15 of 15 (1977 views)

Permalink

> Hi Daniel,
Hi Julien,

>
> >
> > this is the full version of the fdt that threw the error:
> > https://pastebin.com/63TZ9z3k
> > The problematic memory node appears in line 126
>
> Thanks! The output looks corrupted as some of the lines are not valid DTB:
>
> fsl,pins = * 0x000000009300184c [0x00000048];
>
> Although, I am not sure if it is just U-boot dumping the DTB differently.
>
> Anyway, after removing the "corrupted" line, I managed to get a compile
the
> DTB. I don't have a Colibri IMX8QXP. However, given this is an early
parsing
> error, I have just embed the DTB in Xen binary via CONFIG_DTB_FILE.
>
> Unfortunately I couldn't reproduce your error. This either suggests the
DTB gets
> corrupted or Xen doesn't access the DTB with the correct memory attribute.
>
> Do you have the DTB in hand?

Sorry for the corrupted version, I've uploaded the DTB
(fsl-imx8qxp-colibri-eval-v3.dtb) to
https://drive.google.com/drive/folders/1jbpnz35sC0NbCyEjrkLqelBsKBztW1S6?usp
=sharing

I have also uploaded my modified xen source files.
1. arch/arm/bootfdt.c
where I have added the additional printk's seen in the log and
2. arch/arm/setup.c
where I rerun the devicetree parser in line 935 to get the logs, since the
console is not yet initialised when the function is called for the first
time and I
didn't manage to enable earlyprintk.

I think the breaking point is the second memory bank which appears in the
logs (see the output line marked with "!!") with start=0x8 8000 0000 and
size=0.
It isn't specified in the DTB, so I am not sure where this comes from.
It has size=0 so
if ( !size )
{
printk("invalid size, bank %d\n",i);
return -EINVAL;
}
In bootfdt.c makes the function stop.

Log:
(XEN) arch/arm/bootfdt.c: early_scan_node
(XEN) -> fdt: node `memory@80000000': parsing
(XEN) -> process_memory_node
(XEN)
(XEN) arch/arm/bootfdt.c: process_memory_node
(XEN) ->found memory:reg
(XEN) ->cell=
(XEN) ->banks=2
(XEN) ->mem->nr_banks=1
(XEN) ->NR_MEM_BANKS=128
(XEN) ->start=0x80200000 size=0x7fe00000
!! (XEN) ->start=0x880000000 size=0
(XEN) invalid size, bank 1
(XEN) END of arch/arm/bootfdt.c: process_memory_node

Btw 8_8000_0000 is the start address of this systems DDR Main memory,
according to the Reference Manual of the i.MX8QXP.

I hope this helps in reproducing the error.

Cheers,

Daniel Wagner

Mailing List Archive

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files: